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NOTICE 



Version 3.0 of the 1983-84 Public-Use Data Tapes, issued in June 
1985, included a number of reading proficiency scores based upon Item 
Response Theory. These scores were maximum likelihood estimates for 
individuals, each optimal for that individual but not necessarily 
optimal for estimating the distributions of proficiency in the 
population at large or in subpopulations . 

In Version 3.1 of the 1983-84 Public-Use Data Tapes, maximum 
likelihood estimates of reading proficiency scores have been replaced 
with "plausible values" (see Chapter IV, Section F). In addition, 
writing proficiency plausible values have been added to the tapes and 
are explained in Chapter IV, Section G. 

The NAEP 1983-84 Technical Report, cited in this Guide, will be 
published by Educational Testing Service in September 1986. 
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USING THE HABP 1983-84 PUBLIC-USE DATA TAPES 



All data contained on the NAEP 1983-84 public-use data tapes 
have been carefully checked and prepared for general use* The 
basic data consist of student responses to reading and writing 
exercises as veil as to a large number of questions regarding the 
students' backgrounds, attitudes, and activities* Basic data also 
include information about the students' schools and teachers* 

It is important to note that the comple:dty of the NAEP 
design necessarily affects users of the public-use data tapes. It 
is important that users understand the design before performing 
analyses vith the tape* For example, 1983-84 NAEP data were 
collected by both ages and grades; thus, the user must decide 
whether age or grade samples are appropriate for a particular 
analysis* 

The data tapes contain sampling weights for each student* 
These weights should be used in statistical analyses* In 
addition, because of the complex sampling scheme and the 
proficiency estimation technology used in NAEP, conventional 
methods of standard error estimation cannot be applied* The NAEP 
sampling scheme also reduces the effective degrees of freedom for 
statistical analysis* Each of these points is discussed in 
further detail in Chapter V, "Issues to Consider in Conducting 
Statistical Analyses Using 1983-84 NAEP Data*" 

The responses to reading and writing items are available for 
analysis, and these data may be used in any way deemed appropriate 
by the user* These item scores, along with the variables required 
tj compute jdckknifed standard errors of proportions-correct, are 
comparable to the data supplied by ECS on previous NAEP user 
taper* 

The public-use data tapes also contain "plausible values" 
(described in Chapter IV, Section F) of reading and writing 
proficiency for each student for whom calibrated reading or 
writing items are available* These plausible values are 
computational byproducts of the analyses performed for the NAEP 
reports. The Reading RgPgrt Card: Trends in Reading over Four 
National Assessments 1971-1984 (1985) and ^^f^m^S Report Card: 
Writing Achievement in American Schools, 1984 (?<>86)7 Thi 
plausible values yield consistent estimates of the marginal 
effects of the traditional NAEP reporting variables* However, 
analyses involving other effects are subject to biases of varying 
degrees* Approximations of these biases and procedure s for 
correcting for bias are described in the NAEP Technical Report. 
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BIBLIOGPAPEIC CITATION 



National AssessMnt of Educational Progress (NA£;P) 
1983-84 Sample Surveys of Grade 4/Age 9; Grade 8/Age 13; and 
Grade ll/Age 17 Students' Achievement and Attitudes in 
Reading and Writing. Conducted by Educational Testing 
Service for the Office for Educational Research and 
Improvement, Center for Statistics (formerly the National 
Institute of Education) and predecessor agencies. 



Statement of Responsibility 

Prepared by Educational Testing Service for the 
Oifice for Educational Research and Improvement, 
Center for Statistics (formerly the National 
Institute of Education). 

Name of Producer and Distributor 

Educational Testing Service: Princeton, New Jersey- 
Date of Production and Distribution 
June 1986. 

Background Information 

The United States Office of Education and the 
National Center for Education Statistics funded the 
Nation 1 Assessment of Educational Progress from 1970 to 
1979. At that time, responsibility was transferred to 
the National Institute of Education. Prior to the 
1980-81 school year, 9-, 13- and 17-year-olds vers 
surveyed annually in various subject areas, and young 
adults ages 26-35 were surveyed lezs frequently; 
however, since that time, budget restraints have 
prompted a shift to biennial data collection 
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Time Coverage 



The data in these files were collected for the NAEP 
1983-84 ai^sessment session. Data collection was 
initiated on October 10, 1983, and concluded on 
May 11, 1984. Grade 8/Age 13 students were sampled 
between October 10, 1983 and December 16, 1983; 
Grade 4/Age 9 students were saunpled between 
January 2, 1984 and March 9, 1984; and Grade 11/Age 17 
students were sampled between March 12, 1984 and 
May 11, 1984. 



Geographic Coverage 

The United States. Office of Business Economics, 
Department of Commerce regions and census divisions 
are provided, but subregional identifiers have been 
suppressed. NAEP has computed data for four 
geographical regions in the United States (see 
Chapter 17). 



Abstract 

These Public-Use Data Tapes are composed of files 
containing information collected during the 1983-84 
National Assessment of Educational Progress of attitudes 
and achievement in reading and writing for 
'^rade 4/Age 9; Grade 8/Age 13; and Grade xl/Age 17 
students. Each student file provides uata for a 
subset of achievement and attitude items and for a 
common set of background items. Additional raw data 
files provide information aDout teacher and school 
participants and about students who were excluded from 
the sample. The tapes also contain SPSS-X and SAS 
control statement files and a catalog file for each raw 
data file. In addition to the Users' Guide, printed 
layouts and codebooks describe each raw data file. 

Note: The word "exercise" refers to a single 
stimulus followed by one or more items, each item 
requiring a response to the stimulus. Data are recorded 
at the item level in the User-Tape files. 



Tape Specifications 



The 1983-84 Public-Use Data Tapes are available with 
the following standard tape characteristics: 

Recording Density: 1600 or 6250 bytes per inch 

Recording Format: EBCDIC or ASCII 

Blocking: Blocked or Unblocked 

Label: IBM Standard, Unlabeled or ANSI 



Inquiries and Tape Orders 

Inquiries concerning the contents or use of the 
1983-84 Public-Use Data Tapes may be forwarded to the 
individuals below: 

Subject-Matter Inquiries 

Douglas Rhodes 
Associate Director, NAEP 
Educational Testing Service 
Rosedale Road 

Princeton, New Jersey 08541 
Tel: 609-734-1464 

Computer-Related Inquiries 
and Tape Orders 

i^orma Norris 

Senior Research Data Analyst 
Educational Testing Service 
Rosedale Road 

Princeton. New Jersey 08541 
Tel: 609-734-5898 

Tapes may be ordered for all grade/age groups or 
for individual grade/age groups. Orders will be shipped 
with machine-readable dsta files, the Users' Guide, 
printed layouts and codobooks, microfiche copies of 
item texts and a printout of the first 50 records for 
each data file. 
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I. NAEP: An Overview 



A. Introduction 

The National Assessment of Educational Progress (NAEP) is a 
continuing, congressionally mandated national survey of the knowledge, 
skills, understandings, and attitudes of young Americans in major 
learning areas usually taught in school. Its primary goals are to 
detect and report the current status of, as well as changes in, the 
educational attainments of young Americans, and to report long-term 
trends in those attainments. The purpose of NAEP is to gather 
information which will aid educators, legislators and others in 
improving the educational experience of youth in the United States. It 
is* the first ongoing effort to obtain comprehensive and dependable 
achievement data on a national basis in a uniform, scientific manner. 

B. Overview 

Between 1964 and 1969, initial assessment planning and development 
activities were conducted for NAEP with support from both the Carnegie 
Corporation and the Ford Foundation. During this time, objectives and 
exercises were developed for many of the learning areas, sampling and 
data collection strategies were planned, and data analysis plans were 
formulated and outlined. 

From its inception, NAEP has developed assessments through a 
consensus process. In the process, educators, scholars, and lay 
persons design objectives for each learning area, proposing general 
goals they feel Americans should achieve in the course of their 
education. After careful reviews, the objectives are given to item 
writers, who develop measurement instruments appropriate to the 
objectives. 

After the items pass extensive reviews by subject-matter 
specialists, measurement experts, and lay persons, they are 
administered to a stratified multi-stage national probability sample. 
The young people sampled are selected so that assessment results may be 
generalized to the entire national population (see Chapter III for 
details on assessment design). 

NAEP collected data for the first time in 1969. Since that time, 
samples have included over one million 9-, 13- and 17-year-old students 
and, as funding would allow, 17-year-olds who had left school and 
adults 26 to 35 years of age. As Table I.l illustrates, assessments 
have focused on traditional learning areas such as reading, writing, 
mathematics, and science, and on less traditional areas such as 
citizenship, art, literature, music, and career and occupational 
development . 

Since 1971, NAEP has been solely supported by federal funds. 
Funding agencies have included the Office of Education and the National 
Center for Education Statistics (NCES). NAEP is currently supported by 
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the Office for Educational Research and Improvement, Center for 
Statistics (formerly the National Institute of Education)* 

NAEP was administered by the Education Commission of the States 
(ECS) through 1982* In 1983, however. Educational Testing Service 
(ETS) assumed responsibility for administration of the project. In 
assuming responsibility for NAEP, ETS has incorporated an updated 
sampling design (see Chapter III) and, at the same time, has made a 
concerted effort to ensure continuity with previous assessments. 

Public-use data tapes were first produced in 1975, allowing 
outside researchers access to the NAEP database. The 3.0 version of the 
public-use data tapes, produced in June 1985, was the first to be 
produced by ETS, and contained data for the 3983-84 assessment session. 
The current version of the tapes, Version 3.*, also contains 1983-84 
assessment data, but replaces maximum likelihood estimates of reading 
proficiency scores with plausible values, and includes plausible values 
for writing proficiency (see Chapter IV, Sections F and G for more 
information about plausible values). Although it will not be possible 
to change the format of the tapes issued prior to the 1983-84 
assessment session (Versions 1.0 through 2.2), the format of the 
Versions 3.1 and 3.0 tapes has been refined to enhance the ease of use. 
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Table I.l 

National Assessment of Educational Progress 
Learning Areas, Grades and Ages Assessed 
1969-1984 



a<;<;es<;ment year 


LEARNING AREAS 


GRADES/AGES ASSESSED* 


bUaOe 


Age 

Q 


Q 
0 


Age 


viL due 
1 1 


age 

1 / i.O 


Aero 

Age 


Agtp 
ADULT 


Year 1/1969-70 






X 




X 




X 


X 


X 


Writing 




X 




X 




X 


X 


X 


Citizenship 




Y 




Y 
A 




Y 
A 


Y 
A 


y 

A 


Year 2/1970-71 


RpAf) i nor 




Y 
A 




Y 
/\ 




Y 
/\ 


X 


X 


Li terature 




Y 
A 




Y 
/\ 




Y 
i\ 


X 


X 


Year 3/1971-72 


Mus ic 




Y 
A 




Y 




Y 


Y 


X 


Social Studies 




X 




X 




X 


X 


X 


Year 4/1972-73 


Science (2) 




X 




X 




X 


X 


X 


Mathematics 




Y 
A 




Y 
A 




Y 
A 


Y 


X 


Year 5/1973-74 


Career and 
OrriiDa t i onal 
DpvpI nnmpnf 




X 




X 




X 


X 


X 


Writing (2) 




Y 
A 




Y 
A 




Y 
A 


Y 
A 




Year 6/1974-75 


RpflHincr (9^ 




Y 
A 




Y 
A 




Y 
A 


y 

A 




Art 

nx L 




Y 
A 




Y 
A 




Y 
A 


y 

A 




Year 7/1975-76 


Pi fi^pncihin/ 

V/ X L X ^ wtIO il X Li / 

Social 
Studies (2) 




Y 
A 




Y 
A 




Y 
A 


Y 




Ma thema t i c<5** 








X 




X 


X 




Year ft/197fi-77 


Science (3) 




Y 
A 




Y 
A 




Y 
/\ 






Basic Life 
Skills** 












y 

A 






Heal th** 

iAcax Lit 














Y 
/\ 




Rnprcrv** 














y 

A 




Rp;)rl 1 ntrick { 














y 

A 




1 pncp** r ^ 














X 




Year 'ilVill-lZ 


Mathematics (2) 




X 




X 




X 






Consumer 
Skills** 












X 






Year 10/1978-79 


Art (2) 




X 




X 




X 






Music (2) 




X 




X 




X 






Writing (3) 




X 




X 




X 






Year 11/1979-80 


Reading (4) 




X 




X 




X 


X 




Literature (2) 




X 




X 




X 


X 




Year 12/1980-81 


No Data Col 


lection 


Year 13/1981-82 


Mathematics (3) 




X 




X 




X 






Citizenship/ 
Social 
Studies (3) 




X 




X 




X 






Science** (4) 




X 




X 




X 






Year 14/1982-83 


No Data Coj 


.lection 


Year 15/1983-84 


Reading (5) 


X 


X 


X 


X 


X 


X 






Writing (4) 


X 


X 


X 


X 


X 


X 







* 17IS denotes 17-year-olds enrolled in public or private schools; 170S denotes 
17-year-olds who dropped out of school or graduated prior to the assessment. 

** Indicates small, special-interest assessments conducted on limited samples at 
specific ages 

( ) Indicates second and subsequent assessments of a learning area 
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II- The 1983-84 NAEP 



A. The Subject Areas 

In 1983-84, NAEP conducted an assessment of attitudes and 
achievement in reading and writing for more than 100,000 students in 
the age and grade categories outlined in this chapter. 

B. The Instruments 

The assessment session incorporated four distinct types of 
instruments: student assessinent booklets, a teacher questionnaire, a 
school/principal questionnaire, and a questionnaire for excluded 
students. 

!• Student Assessment Instruments 

Based upon the Balanced Incomplete Block (BIB) and 
Unbalanced Incomplete Block (UBIB) sampling design outlined in 
Chapter III, 24 blocks of exercises were used to create a 
total of 63 BIB/UBIB Spiral booklets and four Paced Tape 
booklets for each grade/age group of students. Paced Tape 
booklets were used in group administrations to "pace" students 
through booklets with audio recordings. The instructions were 
read by an announcer, while reading passages, items, and 
response choices were read by the student. The taped 
administrations were used in previoi's NAEP assessments and 
were again used in 1983-84 to explore the effects of the 
change from audio recordings to pencil and paper instruments. 

For the 1983-84 session, each Spiral and Paced Tape 
booklet included six minutes of background and attitude i^-ems 
common to all students. These items are general questions 
concerning materials in the home, parental education, etc, 
AddiLional background and attitude items, as well as reading 
and writing items, were spiralled throughout BIB and UBIB 
booklets (see Chapter III for details concerning BIB and UBIB 
spiralling), A subset of these items was included in the 
Paced Tape booklets (see Chapter III), 

2. The Teacher Questionnaire 

In addition to the collection of data on students, NAEP 
gathered information on the curricula and teaching methods 
usad by selected English and Language Arts teachers. The data 
were provided by teachers who completed a nine-page 
questionnaire, which included questions concerning years of 
teaching experience, frequency of writing assignments, 
teaching materials used, and the availability and use of 
computers. 
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3. 



The School Characteristics and Policy Questionnaire 



The School Characteristics and Policy Questionnaire was 
completed by the principal or his/her representative. The 
five-page questionnaire was used to gather information 
concerning school administration, staffing patterns, English 
curriculum, and student services. 

4. The Excluded Student Questionnaire 

This questionnaire was completed by school personnel for 
every student who was selected for inclusion in the NAEP 
sample, but who was unable to respond to items because he or 
she was non-English speaking, educable mentally retarded or 
functionally disabled. The four-page questionnaire was used 
to gather information concerning special education, languag-s, 
and other student programs. 

The Sample 

In planning for the 1983-84 assessment, NAEP personnel decided 
that sampling students by grade as well as by age would enhance the 
utility of the data, since school districts traditionally delineate 
groups of students by grade rather than age. As a result, the 1983-84 
assessment includes three studv^nt cohorts: students who were either in 
the fourth grade or 9 years old; students who were either in the eighth 
grade or 13 years old; and students who were either in the eleventh 
grade or 17 years old (see Table III. 10 for detailed information on 
grade and age ranges). 

The 1983-84 BIB/UBIB Spiral sample represents two overlapping 
samples. The first sample represents students of specified ages (who 
could be in any grade). This sample is comparable to the samples from 
previous NAEP assessments, and is randomly equivalent to samples of 
students who were administered the Paced Tape session in the 1983-84 
assessment. The second sample represents students of specified grades 
(vho could be of any age). 

Data for future assessments will be gathered by both grade and 
age. The 1983-84 sample is briefly summarized by Tabic II. 1 below 
(detailed information concerning grade/age sampling is contained in 
Chapter III). 
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Table II. 1 
The 1983-84 NAEP Sample 



Grade/Age 


Students Who Took 
BIB/UBIB Spiral 
Assessment Booklets 


Students Who 
Took Paced 
Tape Booklets 


Excluded 
Students 


Grade 4/ 
Age 9 


26,087 


5,492* 


1,416 


Grade 8/ 
Age 13 


28,405 


5,158* 


1,448 


Grade 11/ 
-ge 17 


28,861 


6,209* 


1,361 


TOTALS: 


83,353 


16,859 


4,225 



* Paced Tape booklets were administered at age level only. 



The total number of students who took either Spiral assessment 
booklets cr Paced Tape booklets was 100,212. (For detailed information 
concerning the student sample, see Chapter III.) 

Note that combined grade/age data from the BIB/UBIB Spiral sample 
appears as a single file on the Pullic-Use Data Tape. Any analyses 
intended for grade-only or age-only viata require a separation of the 
data found on this file. 

D. Reporting Subgroups 

In keeping with previous assessments, the data for the 1983-84 
assessment include the following reporting subgroups: student gender, 
student race/ethnicity, size and type of community, region, parental 
education, and grade/age. (The procedures used for classifying data 
into these subgroups are outlined in full in Chapter IV.) 

E. Released and Secure Items 

NAEP releases some items for unrestricted public use. However, 
others are kept secure, as they will be readministered in future 
assessments to determine whether the performance levels have increased, 
decreased, or remained the same. In order to preserve the integrity of 
NAEP, it is essential that these items remain secure. For the 1983-84 
assessment, the complete contents of the School Characteristics and 
Policy, Teacher, and Excluded Student Questionnaires will be released, 
as will all of the background and attitude items administered to 
students. However, reading and writing items will be held secure. 
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P* Procedures for Protecting Confidentiality 



There are two distinct confidentiality issues in the NAEP files 
and documentation* First t and as mentioned above, a number of items 
are kept secure for each assessment session* It is crucial to the long 
term utility of NAEP that these items not be published or used in other 
assessments or research projects* At the same time, the utility of the 
data for secondary researchers is seriously compromised when item text 
is unavailable* Thus, microfiche copies of the complete exercises have 
been included* It is requested that users agree to protect the 
confidentiality of secure items by completing the appended 
Nondisclosure Agreement Form and returning it to ETS. 

The second confidentiality issue involves subregional identifiers. 
NAEP has promised not to reveal the identities of respondents, schools, 
districts, or states. In addition, publication of any of these 
identifiers might permit identification of individual respondents, and 
this would violate the Privacy Act. Consequently, all subregional 
identifiers (school, district, county, state, etc.) have been deleted 
from data files. Scrambled Primary Sampling Unit (PSU), S'lhool, and 
respondent identification fields are included on the data files to 
permit unique identification of each record and to permit aggregating 
data across exercise booklets at the school or PSU level. 
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III. Sample Design 
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A. NAEP Redesign 



III. Sample Design 



1983-84 marks the first NAEP assessment to be auministered by 
Educational Testing Service. In conducting its first assessment, ETS 
incorporated significant changes in both the sample design and 
assessment instruments. 

In past assessments, students at three age levels were 
administered a number of different booklets which contained 
mutually-exclusive sets of cognitive items and a common set of 
background items. Test sessions were timed for groups of students who 
were "paced" through booklets by audio recordings. 

Age-level sampling made it difficult to link NAEP results to state 
and local school policies, most of which are based upon grade-level 
information. In addition, the previous design did not allow analysis 
of the relationship between items appearing in different booklets. 

For the current assessment, ETS retained age-level sampling and 
augmented it with grade-level sampling. As a result, the 1983-84 
assessment includes three student cohorts: students who were either in 
the fourth grade or 9 years old; student? who were eithe r in the eighth 
grade or 13 years old; and students who were either in tfie eleventh 
grade or 17 years old. 

The 1983-84 assessment combined Balanced Incomplete Block (BIB) 
and Unbalanced Incomplete Block (UBIB) designs. The item pool for each 
student cohort was divided into 21 single-length and three 
double-length blocks, for a total of 24 blocks of items. 

BIB Spiralling is a procedure by which only a small subset of 
items is given to an individual student, but the subsets are 
administered in such a way that each pair of items is given to a 
nationally representative subsample of students. The NAEP BIB design 
assured that each possible pair of 19 of the single-length blocks 
occurred in at least one booklet, and, as a result, that inter-block 
analyses would be possible. 

The UBIL design incorporated the two remaining single-length 
blocks, two single-length blocks used in BIB, and the three 
double-length blocks. The BIB/UBIB design was used to create 63 
booklets per grade/age. 

Another design feature of the new NAEP was the collection of 
bridge samples (Paced Tape booklets) by administering some of the NAEP 
items with audio recordings and simple matrix s;^mpling as had ^^een done 
in past assessments. The purpose of these samples was to explore the 
effect of the change from audio recordings to pencil-and-paper 
administration and, if possible, to project the results of past 
assessments onto the new scale (see IRT section in Chapter IV). Using 
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some of the items in the BIB and UBIB Spiral booklets, four Paced Tape 
booklets were administered to age-eligible students of each grade/age. 

A testing session lasted approximately one hour* The BIB and UBIB 
Spiral booklets took 48 minutes of actual testing time; Paced Tape 
booklets took approximately 53 minutes* 

A common background section was included in the beginning of every 
booklet. Students were given six minutes to complete this section* 
Booklets 1-57 (BIB) contained the common background section and three 
single-length blocks. Students vere given 14 ?lnutes to complete each 
of these three blocks* Booklets 58-63 (UBIB) contained the common 
background section, one double-length block, and one single-length 
block* For these booklets, students vere given 28 minutes to complete 
the double-length block and 14 minutes for the single-length block* 
All students in a given Paced Tape session vere administered the same 
booklet, vhich included the six-minute common background section* 

B* Constructing Booklets 

1* JIB Booklets (1-57) 

The first 19 of the single-length item blocks vere 
assigned to booklets in threes* This produced 57 distinct 
booklets* Each possible pairing of the 19 blocks appeared in 
one of the booklets^ and each block occurred in nine booklets* 
Block assignments are outlined in the left half of Table 
111*1* 

Block designations vere recoded using a permutation 
mapping of the 19 letters A-T (except I — there is no block I). 
The booklet numbers vere then recoded using a peirmutation 
mapping of the integers 1-57* Finally, the block orders vere 
randomly permuted vithin each booklet* The final design 's 
shovn in the right half of Table 111*1* 

2* U BIB Booklets (58-63) 

Six additional booklets vere constructed to accommodate 
the remaining blocks (tvo of the BIB blocks, J and R, the tvo 
remaining short blocks, Y and X, and the three double-length 
blocks, U, V, and V)* The tvo BIB blocks vere added to the 
UBIB design to provide a link betveen the tvo item block 
types* Block Q vas substituted for block Y in the booklets 
administered to grade 4/age 9 students* The completes UBIB 
sample design is summarized by Table 111*2* 

The BIB/UBIB booklets in vhich each block appeared are 
listed in Table 111*3* 
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Table III.l 



NAEP 1983-84 Booklet Design BIB Spiral Sample 
(19 X 3 X 57 Cyclic Yorden Rectangle) 



Original Design 


Permuted Design 




Item 


Block 




Item 


Block 


Book 


1 


9 

c 




Book 


1 


2 3 


1 


A 


n 

o 




1 


T 


G L 


2 


B 


r 




2 


A 


L P 


3 


C 


D 


T 

yj 


3 


D 


A T 


4 


D 


p 


V 

JX 


4 


C 


S E 


5 


E 


V 

r 


T 
Lj 


5 


C 


A H 


6 


F 


a 


M 


6 


G 


F H 


7 


G 


u 
n 


N 


7 


K 


R N 


8 


H 


T 

\j 


0 


8 


R 


M F 


9 


J 




p 


9 


0 


N L 


10 


K 


T 


0 
u 


10 


F 


D B 


11 


L 


M 


p 


11 


E 


M A 


12 


M 


N 


Q 


12 


S 


II B 


13 


N 


0 


T 


13 


M 


K D 


14 


0 


p 


A 


14 


T 


N J 


15 


P 


Q 


B 


15 


M 


T C 


16 


Q 


R 


c 


16 


C 


L Q 


17 


R 


s 


D 


17 


H 


E R 


18 


S 


T 


E 


18 


C 


P F 


19 


T 




F 


19 


L 


S K 


20 


A 


c 


L 


20 


N 


B E 


21 


B 


D 


M 


21 


N 


C D 


22 


C 


E 


N 


22 


Q 


K H 


23 


D 


F 


0 


23 


L 


H D 


24 


E 


G 


P 


24 


A 


S R 


25 


F 


H 


Q 


25 


L 


J R 


26 


G 


J 


R 


26 


T 


F Q 


27 


H 


K 


S 


27 


C 


K J 


28 


J 




T 


28 


0 


J S 


29 


K 


M 


A 


29 


Q 


0 D 


30 


L 


N 


B 


30 


B 


Q J 


31 


M 


0 


C 


31 


0 


T H 


32 


N 


P 


D 


32 


B 


M L 


33 


0 


Q 


E 


33 


C 


R 0 


34 


P 


R 


F 


34 


G 


0 E 


35 


Q 


S 


G 


35 


S 


Q M 


36 


R 


T 


H 


36 


B 


A 0 


37 


S 


A 


J 


37 


K 


G A 


38 


T 


B 


K 


38 


0 


F K 


39 


A 


D 


H 


39 


P 


S T 


40 


B 


E 


T 


40 


E 


F L 


41 


C 


F 


K 


41 


H 


M J 
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Table III.l 
(continued) 

NAEP 1983-84 Booklet Design BIB Spiral Sample 
(19 X 3 X 57 Cyclic Youden Rectangle) 



Original Design 


Permuted Design 






Block 






Item Block 


0\J\Ji\, 


1 
X 


2 


3 




DOOK. 


1 


2 3 


42 


D 


G 


L 





42 


J 


E D 


43 


E 


H 


M 




43 


F 


J A 


44 


F 


J 


N 




44 


B 


G C 


45 


G 


K 


0 




45 


P 


B K 


46 


H 


L 


P 




46 


S 


F N 


47 


J 


M 


Q 




47 


P 


Q E 


48 


K 


N 


R 




48 


B 


R T 


49 


L 


0 


S 




49 


P 


M 0 


50 


M 


P 


T 




50 


R 


P D 


51 


N 


Q 


A 




51 


G 


R Q 


52 


0 


R 


B 




52 


S 


G D 


53 


P 


S 


C 




53 


H 


P N 


54 


Q 


T 


D 




54 


T 


E K 


55 


R 


A 


E 




55 


M 


G N 


56 


S 


B 


F 




56 


A 


N Q 


57 


T 


C 


G 




57 


G 


J P 



4 
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Table III. 2 



NAEP 1983-84 Booklet Design 
UBIB Sample 





Long 


Short 


Book 


Block 


Block 


58 


U 


J 


59* 


U 


Y 


60 


V 


R 


61 


V 


X 


62 


V 


X 


63* 


V 


Y 



* In Grade 4/Age 9, block Q was substituted for 
block Y in booklets 59 and 63. 
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Table III. 3 



NAEP 1983-84 BIB/UBIB Spiral Sample 
Block-to-Booklet Correspondence 



Block 










Booklet 


: Numbers 








A 


2 


3 


5 


11 


24 


36 


37 


43 


56 




B 


10 


12 


20 


50 


32 


36 


44 


45 


48 




c 


4 


5 


15 


16 


18 


21 


27 


33 


44 




D 


3 


10 


13 


21 


23 


29 


42 


50 


52 




E 


4 


11 


17 


20 


34 


40 


42 


47 


54 




F 


6 




10 


18 


26 


3d 


40 


43 


46 




G 


1 


6 


34 


37 


44 


51 


52 


55 


57 




H 


5 


6 


12 


17 


22 


23 


31 


41 


53 




J 


14 


25 


11 


28 


30 


41 


42 


43 


57 


58 


K 


7 


13 


1^ 


11 


27 


37 


36 


45 


54 




L 


1 


1 




16 


19 


23 


25 


32 


40 




M 


8 


11 


13 


15 


32 


35 


41 


49 


55 




N 


1 


9 


14 


20 


21 


46 


53 


55 


56 




0 


9 


2d 


29 


31 


33 


34 


36 


38 


49 




P 


1 


18 


39 


45 


47 


49 


50 


53 


5? 




Q 


16 


n 


26 


29 


30 


35 


47 


51 


56 


59* 63* 


R 


7 




17 


24 


25 


33 


48 


50 


51 


60 


S 


~ 4 


12 


19 


24 


28 


35 


39 


46 


52 




T 


1 


3 


14 


15 


26 


31 


39 


48 


54 




U 


58 


59 


















V 


60 


61 


















W 


62 


63 


















X 


61 


62 


















Y 


59** 


63** 



















* Grade 4/Age 9 only 

** Grade 8/Age 13 and Grade 11/Age 17 only 
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3. Paced Tape Booklets (64-67) 



Each of the four Paced Tape booklets has three sections: 
a section of common background items, a section of background 
and attitude items, and a section of cognitive items. 

C- Spiralling Booklets (BIB/UBIB) 

The method for spiralling booklets was designed for two purposes: 

1. To achieve a ratio of nine students taking a UBIB booklet to 
two students taking a BIB booklet in order to meet the 
targeted sample sizes in each category; and, 

2. To distribute the booklets across the sample of students so 
that the booklets within a category (BIB or UBIB) would be 
administered in equal numbers and without positional bias. 

The first purpose was accomplished by forming a cycle of 168 
booklets consisting of two sets of BIB booklets (1-57) and nine sets of 
UBIB booklets (58-63). The BIB and UBIB booklets were merged as 
follows: 









1 


2 


58 


3 


4 


59 


5 


6 


60 


7 


8 


61 


9 


10 


62 


11 


12 


63 


13 


14 


58 


15 


16 


59 


17 


18 


19 


60 


20 


21 


61 


22 


23 


62 


24 


25 


63 


26 


27 


58 


28 


29 


59 


30 


31 


60 


32 


33 


61 


34 


35 


62 


36 


37 


38 


63 


39 


40 


58 


41 


42 


59 


43 


44 


60 


45 


46 


61 


47 


48 


62 


49 


5C 


63 


51 


52 


58 


53 


54 


59 


55 


56 


57 


60 


1 


2 


61 


3 


4 


62 


5 


6 


63 


7 


8 


58 


9 


10 


59 


11 


12 


60 


13 


14 


61 


15 


16 


62 


17 


18 


19 


63 


20 


21 


58 


22 


23 


59 


24 


25 


60 


26 


27 


61 


28 


29 


62 


30 


31 


63 


32 


33 


58 


34 


35 


59 


36 


37 


38 


60 


39 


40 


61 


41 


42 


62 


43 


44 


63 


45 


46 


58 


47 


48 


59 


49 


50 


60 


51 


52 


61 


53 


54 


62 


55 


56 


57 


63 



A given BIB booklet, say #1, appears two times in this cycle; a 
given UBIB booklet, say #58, appears nine times. Administering this 
cycle of booklets evenly across the sample of students establishes the 
ratio of nine UBIB booklets to two BIB booklets. 

In a complete cycle of 168 booklets, each of the six UBIB booklets 
will have appeared nine times and each of the 57 BIB booklets will have 
appeared two times. As a result of this spiralling, each of the 24 
blocks of items used in BIB and UBIB booklets will appear the same 
number of times in a complete cycle (except for blocks J and R, which 
are used in both BIB and UBIB booklets at all three grade/age levels, 
and block Q, which was used in place of block Y for the Grade 4/Age 9 
UBIB booklets). 

Each block, except for blocks J, R, and Q, appears exactly 18 
times in the 168-booklet cycle. Blocks J and R appear 27 times. Block 
Q appears 36 times in the Grade 4/Age 9 spiralling cycle. Block Y 
appears 0 (zero) times in the Grade 4/Age 9 spiralling cycle. See 
Tables III. 4, III. 5, and III. 6 for the block administration and the 
actual number of students who were assessed as a result of the BIB and 
UBIB spiralling design. 
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Table III. 4 



NAEP 1983-84 
Number of Blocks Administered: 
BIB/UBIB Spiral and Paced Tape Sample 
Grade 4/Age 9 Students 



Block 


Total 


Weighted N 


Standard Error 
(of Weighted N) 


Coefficient 
of Variation 


BIB/UBIB Spiral (Grade 4/Age 9) 




A 


27/1 


4279zO 


5505 


1.20 


B 


2755 


426861 


4903 


1.15 


C 


2776 


428236 


4311 


1.01 


D 


2790 


423029 


5473 


1.20 


E 


2741 


418120 


4383 


1.05 


F 


2744 


420904 


6210 


1.48 


G 


2804 


427961 


6438 


1.50 


H 


27^5 


425636 


5130 


1.21 


J* 


4213 


641751 


5789 


0.90 


K 


2800 


421737 


3309 


0.78 


L 


2776 


425012 


4789 


1.13 


M 


2702 


427264 


5271 


1.23 


N 


2810 


425105 


5346 


1.26 


0 




427033 


7718 


1.81 


P 


2803 


410350 


7384 


1.76 


Q** 


56ii 


851190 


7867 


0.92 


R*** 


4211 


635696 


6315 


0.99 


S 


2815 


426905 


4235 


0.99 


T 


2788 


424099 


4161 


0.98 


U 


2818 


427632 


3197 


0.75 


V 


2811 


428207 


4620 


1.08 


w 


2790 


427364 


3791 


0.89 


X 


2780 


425005 


4536 


1.07 


Total 
Spiral 


26087 


3971749 


18935 


0.48 


Pacec 


1 Tape (Age 9) 


P64 


1403 


3122045 


22296 


0.71 




1356 


3005769 


31030 


1.03 


P66 


1389 


3087985 


16357 


0.53 


P67 


1344 


3100713 


20866 


0.67 



* Block J appeared in both BIB and UBIB booklets 

** Block Q was substituted for block Y in booklets 59 and 63 

*** Block R appeared in both BIB and UBIB booklets 
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Table III. 5 



NAEP 1983-84 
Number of Blocks Administered: 
BIB/UBIB Spiral and Paced Tape Sample " 
Grade 8/Age 13 Students 



Block 


Total 


Weighted N 


Standard Error 
(of Weighted N) 


Coefficient 
of Variation 


BIB/UBIB Spiral (Grade 8/Age 13) 


A 


3075 


467770 


6525 


1.40 


B 


3042 


464415 


4815 


1.04 


C 


3052 


468125 


5179 


1.11 


D 


3053 


466462 


4825 


1.03 


£ 


3069 


47Z113 


3935 


0.83 


F 


3072 


472426 


3481 


0.74 


G 


3030 


467003 


5836 


1.25 


H 


3046 


mm 


6260 


1.34 


J* 


4525 


691829 


5444 


0.79 


K 


3089 


474683 


3686 


0.78 


L 


3075 


465801 


5362 


1.15 


M 


3057 


470741 


4481 


0.95 


N 


j076 


472133 


5979 


1.27 


0 


2974 


456279 


7782 


1.71 


P 


3075 


470466 


6721 


1.43 


Q 


2982 


457437 


5589 


1.22 


R** 


4524 


699666 


6888 


0.98 


S 


3063 


467995 


5054 


1.08 


T 


3060 


468299 


4252 


0.91 


U 


3043 


470341 


4714 


1.00 


V 


3042 


472259 


4873 


1.03 


W 


3022 


463612 


4665 


1.01 


X 


3033 


467056 


4398 


0.94 


Y 


3029 


468345 


4007 


0.86 


Total 
Spiral 


28405 


4363428 


11280 


0.26 


Facet 


Tape (Age 13) 


P64 


1310 


3310355 


24447 


0.74 


P65 


1276 


3348263 


15850 


0.47 


P66 


1283 


3338947 


17754 


0.53 


P67 


1289 


3340216 


16576 


0.50 



* Block J appeared in both BIB and UBIB booklets 
** Block R " " " " " " " 
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Table 111*6 



NAEP 1983-84 
Number of Blocks Administered: 
BIB/UBIB Spiral and Paced Tape Sample 
Grade 11/Age 17 Students 



Block 


Total 


Weighted N 


Standard Error 
(of Weighted N) 


Coefficient 
of Variation 


BIB/UBIB Spiral (Grade 11/Age 17) 




A 


3098 


434407 


5944 


1.37 


B 




433284 


4421 


1.02 


C 


3084 


433397 


6444 


1.49 


D 


3124 


437093 


4105 


0.94 


E 


3089 


426863 


4567 


1.07 


F 


3082 


431695 


4191 


0.97 


G 


mi 


435683 


4746 


1.09 


H 


3100 


433534 


3556 


0.82 


J* 


4701 


663249 


5079 


0.77 


K 


3080 


430441 


5638 


1.31 


L 


3100 


43610Z 


4483 


1.03 


M 


3045 


427213 


6988 


1.64 


N 


3066 


430797 


4774 


1.11 


0 


3093 


433640 


4743 


1.09 


P 


3120 


435059 


5844 


1.34 


Q 


3083 


436621 


4803 


1.10 


R** 


4641 


649350 


6015 


0.93 


S 


3084 


431261 


4242 


0.98 


T 


3076 


432770 


4213 


0.97 


U 


3109 


437217 


5054 


1.16 


V 


3102 


435765 


4342 


1.00 


W 


3071 


429283 


5091 


1.19 


X 


3056 


427368 


4446 


1.04 


Y 


3080 


430553 


4524 


1.05 


Total 
Spiral 


28861 


4045040 


11103 


0.27 


Paced Tape (Age 17) 


P64 


1539 


3048025 


9151 


0.30 


P65 


1540 


3045282 


1Z441 


0.41 


P66 


1596 


2978520 


30679 


1.03 


P67 


1534 


3026687 


10433 


0.34 



* Block J appeared in both BIB and UBIB booklets 
** Block R " " " " " " » 
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The second purpose was accomplished by collecting this cycle of 
168 booklets into bundles of 23 consecutive booklets, with consecutive 
bundles beginning where the previous bundle left off; the last of the 
168 booklets was always followed by the first in a continuous circling 
process (hence the term "spiralling"). As a result; 168 different 
bundles were created and each booklet distributed evenly throughout 23 
positions in the bundles. By shipping consecutive bundles to schools, 
the likelihood that any given booklet would be used was equalized 
across the sample. 

The appearance of each booklet in each position of an 
administration sequence was also equalized. Table III. 7 lists the 
number of booklets administered; Tables III. 8 and III. 9 contain charts 
of spiralling results. 

Sample Design 

Westat, Inc. and the Research Triangle Institute (RTI) played 
major roles in sample design, selection, and field administration for 
the 1983-84 NAEP assessment. The 1983-84 sample design is described in 
detail in a technical report issued by Westat in 1985. This section is 
intended as a brief overview of the sample design. 

NAEP uses a stratified, three-stage national probability sample 
design. The first and second sample stages were designed by RTI. 
County and school sample selection was made by RTI and monitored by 
ETS/Westat. Final student selection was made by Westat staff as they 
collected data in the schools (see Chapter IV for information 
concerning the role of Westat staff in conducting assessment sessions). 
Briefly, the sample design consisted of che following: 

1. Regions/Primary Sampling Units 

In the first stage of sampling, the United States was 
divided into geographical units comprised of counties, or 
groups of contiguous counties, which met a minimum school 
enrollment size. These units, called Primary Sampling Units 
(PSUs), were stratified by both region and size of community. 
A sample of 64 PSUs was then selected (with replacement) to 
represent all regions and sizes of communities with 
probability proportional to population size measures. 

2. Schools 

In the second stage of sampling, all public, private. 
Catholic, Bureau of Indian Affairs, and Department of Defense 
schools were listed according to the three grade/age groups 
within each of the 64 PSUs. 

To allow sampling of extreme-low Socioeconomic Status 
(SES) big-city schools at a rate twice that of all others, 
schools were stratified by SES within hig-city PSUs. 
Extreme-rural schools were also oversampled by a factor of two 
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Table 111*7 

NAEP 1983--84 Number of Booklets Administered 
BIB/UBIB Spiral and Paced Tape Sample 



Booklet 
Number 


br* 4/Age y 




KjtL* 11/ Age 1 / 


1 






J*f 0 


2 


71 1 
Jll 




1^1 
JOJ 


3 


71 £ 
Jib 


1/. 1 


1SS 
J J J 


4 


Ol o 

J19 


i/.n 


ISA 
J J*f 


c 


J09 




11Q 
J J7 


c 
0 




111 
J Jl 


1A1 
J*f 1 


7 


11 "7 

31/ 


J Jj 


IIS 
J JJ 


8 


JOo 




111 
J J J 


9 


Ol Q 

jio 


11Q 
J JO 


11A 
J JO 


10 


iOo 


1 /. /. 


1A0 


11 


309 


1 /. 1 

j4j 


107 
JZ / 


IZ 


j09 


1/.G 


117 
J J / 


13 


J09 


1 *;i 
JJJ 


19A 


14 


jUo 


1 /. /. 


lAH 


1.) 




1/. 1 


111 
J J J 


lb 


in«; 


11Q 


117 
J J / 


1 / 


jUo 


11A 
J JO 


lAH 


1 O 

lo 




1AQ 


llfl 
J JO 


19 


JUo 


1S7 
JJ / 


lAH 






1A1 

Jif J 


lAO 


^1 


11 0 

JlZ 


11A 

J Jo 


lAH 


00 

11 


11 1 


117 
J J / 


J** / 


0 Q 

^3 




10Q 
J^O 


■^SA 
j-j** 


24 


307 


332 


338 


25 


317 


328 


336 


26 


312 


325 


344 


27 


315 


327 


350 


28 


329 


322 


347 


29 


319 


328 


349 


30 


317 


314 


350 


31 


307 


324 


338 


32 


316 


332 


345 


33 


306 


331 


344 


34 


296 


328 


344 


35 


302 


335 


340 


36 


311 


336 


344 
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Table III. 7 
(continued) 



NAEP 1983-84 Number of Booklets Administered 
BIB/UBIB Spiral and Paced Tape Sample 



Booklet ' 








Number 


Gr. 4/Aee 9 Gr 


. 8 /Age 13 


Gr. 11/Age 17 


37 


303 


336 


350 


38 


306 


342 


345 


39 


306 


346 


335 


40 


291 


352 


343 


41 


308 


346 


348 


42 


290 


345 


359 


43 


292 


344 


353 


44 


305 


342 


349 


45 


317 


352 


345 


46 


313 


344 


343 


47 


314 


332 


338 


48 


310 


330 


341 


49 


317 


325 


348 


50 


309 


338 


351 


51 


315 


323 


349 


52 


322 


338 


350 


53 


316 


349 


356 


54 


312 


350 


344 


55 


316 


338 


347 


56 


313 


349 


329 


57 


317 


339 


346 


58 


1422 


1516 


1572 


59 


1396 


1527 


1537 


60 


1416 


1529 


1574 


61 


1395 


1513 


1528 


62 


1385 


1520 


1528 


63 


1405 


1502 


1543 




Paced 


Books 




64 


1403 


1310 


1539 


65 


1356 


1276 


1540 


66 


1389 


1283 


1596 


67 


1344 


1289 


1534 


Total 


31579 


33563 


35070 


Total BIB/UBIB 26087 


28405 


28861 


Total Paced 5492 


5158 


6204 
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Table III. 8 

NAEP 1983-84 BIB Spiral Sample 
Number of Students per Block 



tfE 9 / GRADE < (TOTAL N' 21,35?) 




«3CDEFCHJKLNN0P0RSTUVUX 
BLOCI^ 

PCE 13 / GRADE 8 (TOTW. N • n.m) 




A3CDEFCHJKLHN0P0RSTUVUX i 
BLOCK 

AGE 17 / GRADE 11 (TOTft N -■ 'ii.iili 



\ — 
Co 

of 



3 

is ^ 



i i 



3g 3 

3a il 



hSCDEF&HJ|CLHNOPORSTU/U;;i 
BLOCK 



ERIC 
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Table III. 9 



NAEP 1983-8A BIB/UBIB Spiral and Paced Tape Sample 
Number of Students per Book 



«E 9 / GRADE < (TOR N = 3!,5?9) 



UJ 

i m- 



3 



i 



BIB SPIRWiD 



II 21 31 41 51 

BOOKLET HJHBER 



Ufilfi 



PACED 



5S b( 



AGE 13 / GRADE 8 (TOTfl. N = 53, 503) 




It 48 

BOOKLET MJMBER 



fiE 1? / GRADE 11 (TOTfL N = 35,?70) 



1,488- 



§1,283- 



I 588- 



BIB SPIRW.ED 



=^ .Mil 



UBI5 ?ftCED 



|.)»JlMI« 



m 

I 

>1 



ilJi 



X 



19 



21 



39 £9 

BOOflET mfBER 



5P 



55 



b4 



ERLC 
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within extreme rural ?SUs» Extreme-low SES and extreme-rural 
schools were oversampled to insure adequate sample sizes > 
thereby enhancing the reliability of estimates. 

Schools within each PSU were selected (without 
replacement) with probabilities proportional to assigned 
measures of size* Roughly equal measures of size were 
assigned to schools containing estimates of age-eligible 
students ranging from 20-160 (for age 9), or 20-200 (for ages 
13 and 17). Schools above the indicated maximum size were 
selected with probabilities proportional to the number of 
age-eligible students* Schools with less than 20 estimated 
age-eligibles were assigned considerably lower measures of 
size, since they had higher peir-student administrative costs. 

Of the resulting total original sample of 1,705 schools, 
there were 197 refusals at the school and district levels, and 
147 schools were eliminated because they were out-of -range, 
closed or had no eljgibles enrolled. This resulted in 1,361 
cooperating schools , to which a number of schools were added 
to replace the refusing schools. The total number of schools 
on the Public-Use Data Tapes files is 1,430; in several of 
these, no eligible students were selected. Assessments vere 
conducted in 1,465 sch^^ols. 

3. Students 

In the third stage of sampling, a consolidated list of 
all grade-and-age-eligible students was established for each 
selected school. A systematic selection of eligible students 
was made, students were assigned to Spiral Paced Tape 
sessions, and assessment booklets were admii tered to 
selected students. 

Of the 104,437 students sampled for the 1983-84 
assessment, 4,225 were exluded from participation because they 
were judged by school personnel to be educable mentally 
retarded, non-English speaking, or functionally disabled. 
Spiral sessions were administered to 83,353 students; Paced 
Tape sessions were administered to 16,859 studenl"^. Tabie 
III. 10 provides information concerning grade/age ^overa^^e. 

4. Questionnaires 

The School Characteristics and Policy, Excluded Student, 
and Teacher Questionnaires were distributed in every sampled 
school. If the forms were completed promptly, the supervisor 
returned them to ETS. The School Characteristics and Policy 



*A cooperation rate of 88.1 wa^ achieved. See the NAEP 
1983-84 Technical Report for the rate calculation method. 
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Table III. 10 



NAEP 1983-84 
Number of BIB/UBIB Students by Grade/Age 



Grade 4/Age 9 Students 





AGE 




< 9 


= 9 


> 9 


TOTAL 


GRADE < 4 










UNWEIGHTED N 


0 


5917 


0 


5917 


WEIGHTED N 


0 


761524 


0 


761524 


STANDARD ERROR 




8099 




8099 


COEFF. OF VAR. 


- 


1.06 


- 


1.06 


GRADE = 4 










UNWEIGHTED N 


158 


12953 


6984 


20095 


WEIGHTED N 


21504 


2295588 


883198 


3200290 


STANDARD ERROR 


2395 


9067 


7089 


14047 


COEFF. OF VAR. 


11.14 


0.39 


0.80 


0.44 


GRADE > 4 










UNWEIGHTED N 


0 


75 


0 


75 


WEIGHTED N 


0 


9936 


0 


9936 


STANDARD ERROR 




2163 




2163 


COEFF. OF VAR. 




21.77 




21.77 


GRADE TOTAL 










UNWEIGHTED N 


158 


18945 


6984 


26087 


WEIGHTED N 


21504 


3067047 


883198 


3971749 


STANDARD ERROR 


2395 


13693 


7089 


18935 


COEFF. OF VAR. 


11.14 


0.45 


0.80 


0.48 
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Table III. 10 
(continued) 

NAEP 1983-84 
Number of BIB/UBIB Students by Grade/Age 



Grade 8/Age 13 Students 





AGE 




< 13 


= 13 


> 13 


TOTAL 


GRADE < 8 










UNWEIGHTED N 


A 
\J 




U 




WEIGHTED N 




1 n'^/i 711 


f\ 
\J 


1Uj4 /II 


STANDARD ERROR 




6087 




6087 


COEFF. OF VAR. 




0.59 




0.59 


GRADE = 8 










UNWEIGHTED N 


184 


14515 


7151 


21850 


WEIGHTED N 


25662 


2269841 


1018446 


3313949 


STANDARD ERROR 


4070 


4025 


6860 


7824 


COEFF. OF VAR. 


15.86 


0.18 


0.67 


0.24 


GRADE > 8 










UNWEIGHTED N 


0 


60 


0 


60 


WEIGHTED N 


0 


14769 


0 


14769 


STANDARD ERROR 




4144 




4144 


COEFF. OF "AR. 




28.06 




28.06 


GRADE TOTAL 










UNWEIGHTED N 


184 


21070 


7151 


28405 


WEIGHTED N 


25662 


3319320 


1018446 


4363428 


STANDARD ERROR 


4070 


8088 


6860 


11281 


COEFF. OF VAR. 


15.86 


0.24 


0.67 


0.26 
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Table III. 10 
(continued) 

NAEP 1983-84 
Number of BIB/UBIB Students by Grade/Age 



Grade 11/Age 17 Students 





AGE 




< 17 


= 17 


> 17 


TOTAL 


GRADE < 11 










UNWEIGHTED N 


0 


4129 


0 


4129 


wEIGBTED N 


0 


671683 


0 


671683 


STANDARD ERROR 




21099 




21099 


COEFF. OF VAR. 










GRADE =: 11 










UNWEIGHTED N 


2386 


16787 


3692 


22865 


WEIGHTED N 


399289 


2037738 


635595 


3072622 


STANDARD ERROR 


22106 


3439 


21691 


6816 


COEFF. OF VAR. 


5.54 


0.17 


3.41 


0.22 


GRADE > 11 










UNWEIGHTED N 


0 


1867 


0 


1867 


WEIGHTED N 


0 


300736 


0 


300736 


STANDARD ERROR 




20072 




20072 


COEFF. OF VAR. 




6.67 




6.67 


GRADE TOTAL 










UNWEIGHTED N 


2386 


22783 


3692 


28861 


WEIGHTED N 


399289 


3010157 


635595 


4045041 


STANDARD ERROR 


22106 


8221 


21691 


11104 


COEFF. OF VAR. 


5.54 


0.27 


3.41 


0.27 
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Questionnaire was mailed to the school by Westat prior to the 
assessment. All other questionnaires were distributed by the 
supervisor at the time of the assessment. 

An Excluded Student Questionnaire was completed for every 
student who was selected but had to be excluded from the 
assessment. A subgroup of students sampled through Spiral 
sessions was identified for administration o£ tlie Teacher 
Questionnaire. This subsample was equal to the number of 
Spiral sessions conducted in the school. Thus, if there were 
six Spiral sessions conducted in a school, a subsoinple of six 
students was selected and the school coordinator was asked to 
identify the English or Language Arts instructor for each 
student. These instructors completed the Teacher 
Questionnaires. Please note that, as a number of students may 
have had the same teacher, and some teachers did not complete 
the questionnaire, the number of students in the subsample is 
greater than the number of teachers vho completed 
questionnaires in a given school. 

Overall, 88.7 percent of the School Characteristics and 
Policy Questionnaires, 92.7 percent of the Excluded Student 
Questionnaires, and 86.9 of the Teacher Questionnaires were 
completed and returned to ETS. 

The Principal Questionnaire, distributed to each sampled 
school by Westat prior to the assessment, was used by Westat 
to determine an estimate of grade/age eligible students. 
(Principal Questionnaire data is not available on the 
Public-Use Data Tapes.) 

Sampling Weights; Caveats 

NAEP uses differential sampling rates. Some 
subpopulations are deliberately oversampled in order to obtain 
larger samples of respondents in various reporting groups. 
Also, adjustments for nonresponse introduce differential 
weights. The use of post-stratification and certain other 
design features is also reflected in the weights that are 
provided (see Sections 6 and 7 below). Consequently, sampling 
weights are supplied on the data files so that each respondent 
can be represented in the desired proportion in data analyses. 

a. Oversampled Subpopulations 

As a result of oversampling, certain 
subpopulations, which have different characteristics 
from the rest ol the population, are overrepresented 
in the sample. Analyses which ignore this 
overrepresentation may produce biased and misleading 
data, since the subpopulations may have unwarranted 
impact on results. Consequently, NAEP sampling 
weights should be used for all analyses, whether 
exploratory or confirmatory. 
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Age Population 



The saraple of students responding to exiy one of 
the four Paced Tape booklets assigned to a given age 
is a representative sample of the population of all 
students of that age. Consequently, an estimate of 
the total number of age-eligible students in the 
population can be obtained oy summing the saunpling 
weights of all students (including excluded students) 
selected to receive a specified Paced Tape booklet. 
Since there are four Paced Tape booklets, there are 
four separate estimates of the total available. 
These totals are shown in Tables III. 4, III. 5 and 
III. 6. Another estimate would be the average of 
these four estimates. 

Similarly, an estimate of the total of a given 
subpopulation of age-eligible students (€:.g.t 
9-year-old Hispanic students) may be obtained by 
summing the sampling weights for that subpopulation 
of all such students (including excluded students) 
selected to receive any given one of thn Paced Tape 
booklets. A better estimate of this total is the 
average of the four totals, one from each Paced Tape 
booklet. 

The sample of students who responded to any of 
the Spiral booklets who are also of the given age is 
also a representative sample of the population of 
age-eligible students. Consequently, an estimate of 
the total number of age-eligible students can be 
obtained by summing the weights for all students 
(including excluded students) responding to any of 
the Spiral booklets who were also in the specified 
age class. Estimates of subpopulation totals can be 
obtained by further restricting the summation to 
students who are additionally in the desired 
subpopulation. 

Grade/Age and Grade Populations 

An estimate of the total population of 
grade/age-eligible students can be obtained by 
summing the sampling weights of all the students 
(including excluded students) who responded to any of 
the Spiral booklets. An estimate of the total 
population of grade-eligible students can be obtained 
by summing the weights of all students (including 
excluded students) responding to any Spiral booklet 
who were also in the modal grade (i.e., grade 4, 8, 
11). 
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d» Weighted N 



Please note that the "Weighted N" appearing next 
to the total number of administrations of each of the 
BIB/UBIB blocks in Tables 111*4, III. 5 and III. 6 is 
not an estimate of the national population of 
grade/age eligibles. Weighted N is an estimate of 
the total number of grade/age eligibles who vould 
have been assigned the block if every grade/age 
eligible student in the nation had been assessed 
using the BIB spiralling design* This estimate is 
roughly one-ninth of the total national population of 
grade/age eligibles for all blocks except those which 
appeared in both BIB and UBIB booklets. 

e. Standard Errors 

The sampling weights are statistics and are 
therefore subject to variability. Standard errors of 
any population estimates (such as those indicated in 
the above paragraphs) can be computed using the 
jackknife procedures described in Chapter IV. 

The standard error of the weighted N shown in 
Tables 111*4, 111*5 and 111*6 were computed using the 
jackknife procedures. The column headed "Coefficieat 
of Variation" presents the standard error as a 
percent of the weighted N. 

Weights Contained on the Public-Use Data Tapes 

NAEP has provided two types of weights for the 1983-84 
assessment: full-sample weights and replicate weights used 
for variance estimation. Both the full sample and replicate 
weights include adjustments for nonresponse and 
post-stratification, 

a. Student Weights 

1. Full-Sample Weig ht: 

- Student rull-sample weight (WEIGHT): Use 
for analysis at the student level 

The following weights should not be 
used for student analysis. These weights 
were used in deriving the final student 
full-sample weight (WEIGHT): 

- PSU weight (PSUWT) 

- School weight conditional on PSU (SCHWT) 

- Session weight conditional on school 
(SESSWT) 
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- Student base weight conditional on session 
type (STUDWT) 

Computation of the student full-sample 
weight is described in Section 7 below. 

2. Replicate Weights ; 

- Jackknife replicate weights 01 to 32 
(JKWTxx): Use these weights for computing 
jackknife variance estimates at the 
student level 

Use and computation of these weights 
are described in Chapter IV, Section H. 

Teacher-Student Weights 

Note: The teacher-student weights are 
appropriate for use in estimating the number or 
percent of students in the total population who have 
various characteristics; they are not appropriate for 
use in estimating the number of teachers, or the 
number of teachers with various characteristics. 
They are supplied only for a subsample of students 
who were selected from the full NAEP sample and are 
appropriate for use in analyses involving the 
teacher-characteristics of students. For example, 
these weights would be used to estimate the 
proportion of students who have Hispanic language 
arts teachers. 

1. Full-Sample Weight : 

- Final r.eacher-student weight (TSTUWTF): 
Use for analyses when teachers' 

' characteristic:; are linked to students 

The following weights should not be 
used for teacher analysis. These two 
within-school weights were used in deriving 
the final teacher-student weight (TSTUWTF): 

- Teacher-student weight adjusted for 
nonresponse (TSTUWTN) 

- Teacher sel^^ction weight (TSELWT) 

2. Replicate Weights : 

- Teacher-student replicate weights 01 to 32 
(TSWTxx): Use these weights for computing 
jackknife variance estimators when 
teachers' characteristics are linked to 
students 
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Excluded Student Weights 



1, Final Weigh t t 

- Excluded student final weight (XwEIGHT): 
Use for analysis of the Excluded Student 
Questionnaire data 

The following weights should not be 
used for excluded student analysis* These 
three weights were used to derive the final 
excluded student weight (XVEIGHT): 

- PSU weight (XPSUWT) 

- School weight conditional on PSU (XSCHVT) 

- Within-school student weight (WSEXWT) 

2. Replicate Weights : 

- Jackknife replicate weights 01 to 32 
(EXWTxx): Use thest ^eights for computing 
jackknife variance estimators for the 
Excluded Student Questionnaire data 

School Weights 

1. Full Sample ; 

Note: To weight and analyze (at the 
school level) the School Questionnaire data, 
multiply the following weights: 

Final Weight = SCHNRFS x SPSUWT x SSCHVT 

- School nonresponse factor, Spiral 
(SCHNRFS) 

- PSU weight (SPSUTT) 

- School weight conditional on PSU (SSCHVT) 

The following weights should not be 
used for school analysis. These weights 
were used to derive the final school 
full-sample weight. 

- Overall school weight. Paced Tapes 
(SCHWTT) 

- School session weight (SVTSPI) 

- School session weight , specific Paced Tape 
session (SWTTAP) 

- Overall school weight. Spiral (SCHVTS) 
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Weighting Procedures 

This section contains additional information about the 
procedures used by NAEP to derive weights. Readers who do no / 
need detailed technical information concerning the derivation 
of weights may disregard this section of the guide. 

The standard full-sample weight (WEIGHT) consists of the 
student's base weight multiplied by the various adjustments 
described in Sections a through g below. 

The base weight assigned to a student is the reciprocal 
of the probability that the student was invited to a 
particular type of assessment session, i.e., a Spiral session 
or a particular Paced Tape session. That probability is the 
product of four factors: 

1. The probability that the PSU was selected 
(PSUWT); 

2. The conditional probability, given the PSU. 
that the school was a member of the sample 
selected by RTl or any supplementary sample 
selected by Westat (SCHWT); 

3. The conditional probability, given the 
sample of schools in a PSU, that the school 
was allocated the specified type of session 
(SESSWT); and 

4. The conditional probability, given the 
school, that the student was invited to the 
specified type of session (STUDWT). 

Thus, the base weight for a student may be expressed as 
the product 



W = W^ . Wj • . w^ 



where 




PSU weight; 



school weight conditional on the PSU; 



the reciprocal of the conditional 
probability, given the sample of schools, 
that the school is allocated a specified 
type of session; and 
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s the reciprocal of the wi thin-school 

selection probability for students sampled 
for Spiral or a specific Paced Tape 
session. 

The PSU weight, W , was provided by RTI, When partial 
PSU replacements were identified (see technical report issued 
by Westat in 1985), the PSU weight associated with the schools 
in the replacement county(ies) was that of the originally 
selected PSU. 

The school weight, W,, was provided by RTI in a computer 
file of sample schools. The computation of the probability of 
selection for schools in the supplementary sample selected by 
Vestat whenever the original sample turned out to be too small 
is discussed in the Vestat technical report, as is the 
computation of the school weight for substitute schools. 

The session allocation weight, V^, was computed by 
enumeration of all possible allocations yielded by the 
algorithm used by Vestat to allocate Paced Tape and Spiral 
sessions to sample schools or school clusters (see the Vestat 
technical report). 

For Spiral sessions, the wi thin-school student weight, 
V^, IS simply the sampling interval for selecting students for 
Spiral sessions. For Paced Tape sessions, the wi thin-school 
student weight accounts for whether or not there was Spiral 
sampling in the school and the conditional sampling interval 
for Tape. 

a. Adjustment of Base Veights for Nonresponse 

The base weight for a student was adjusted by 
two nonresponse factors: one to adjust for 
non-cooperating schools and one to adjust for 
students that were invited to the assessment but did 
not appear either in the scheduled session or in a 
makeup session. Thus the within-PSU nonresponse 
adjusted weight was of the form 



V ^ V f W V f 

wherci the nonrespoiise adjustment factors, f^ and f^^ 
were computed as described below. 

b. School Nontx^sponse Adjustment 

School nonresponse factors were computed 
separately within each PSU for up to two or three 
classes of schools using as many nonresponse classes 
as the number of sampled schools in the PSU and 
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nonresponse pattern alloved. However ^ since it was 
required that each class contain at lease four or 
five schools, often only one class was identified in 
the PSU. 

For any nonresponse class, c, the school 
nonresponse factor for Spiral sessions is given by 



'Ic 



isA 

i€B 



where 

= school weight (the reciprocal of the 
probability of selection of the school 
conditional on the PSU); 

= estimated number of grade-eligible 

students in school i based on QED data 
(:5ee Chapter IV, Section J) and/or the 
Principal Questions Ire; 

set A consists of the original sample of 

eligible schools (including supplemental, 
new and refusing schools, but not 
substitutes); and 

set B consists of all cooperating schools 

(including schools that were substituted 
for non-cooperating schools). 



Note that, for a substitute school, was 
defined as the weight of a school with the same 
measure of size as if it had been selected by the 
probability selection procedure. 

A school nonresponse adjustment was applied to 
the base weight of students in Spiral but not Paced 
Tape sessions, as the four required Paced Tapes per 
PSU were always allocated to the cooperating schools 
in a PSU. As a result, only weights for Spiral 
sessions were affected by school nonresponse. 

Student Nonresponse Adjustment 

Student nonresponse adjustment factors were 
computed separately for Spiral sessions and for each 
of the four Paced Tape sessions within each PSU. 
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d. Nonresponse Adjustment for Students in Paced Tape 
Sessions 

For each Paced Tape session^ t, in a PSU, the 
nonresponse factor f^^ was computed by 



'■2t 



where 

n^ = number ol students invited to the 

particular Paced Ta( session in the PSU; 
and 

n' ■ number of students who completed the 
session. 

Note that in the common situation where all 
students invited to a Paced Tape session were from a 
single school^ no school weight (such as appears 
below in the adjustment factor for Spiral sessions) 
is needed to compute the nonresponse adjustntent 
factor; the weighted ratio equals the unweighted 
ratio. In the occasional situation where a school 
cluster was involved, it would have been appropriate 
tu introduce the school wei;;ht in the adjustment. 
This was not done because of the infrequent 
occurrence of school clusters, and because the 
aggregate effect of applying the school weights in 
such cases vculd have been only marginally different 
from the adopted procedure of using the ratio n^/n^ . 

e. Nonresponse Adjustment for Students in Spiral 
Sessions 

For Spiral sessions, the student nonresponse 
adjustment was maue separately for two classes of 
students: those in or above the modal grade for 
their age and those below the modal age. 

The factor for students in class c in a 
particular PSU was computed by 



^2c = _i 



Z w.n, 

i i c 

1 



where the summations extend over the schools in the 
PSU and 
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"^ic number of Spiral invited students in 
school i and student class c; 

n^^ a number of Spiral tested students in 
^ school i and student class c; and 

a the reciprocal of the probability of 
assignment of a student in school i to a 
Spiral session, conditional oa the PSU| 
adjusted for school nonresponse* 

Adjustment for Missing Paced Tape Ses sions 

In a few instar^ces, the supervisor inadvertently 
administered Spiral booklets rather than the assigned 
Paced Tape booklet* Jr, a school which was allocated 
a Paced Tape session refused just before the 
assessment was conducted, without prov'.Jing enough 
time to reassign the Paced Tape session to another 
school* This occurred in seven of the 768 Paced Tape 
sessions assigned to the three grade/age groups* 

The following imputation procedure was used to 
deal with this type of nonresponse. For variance 
computation purposes, the 64 NAEP PSUs had been 
grouped into 32 pairs. Let the PSU requiring the 
imputation be called the "recipient" PSU and the 
other member of the same pair the "donor" PSU. A 
one-half subsample of the students administered the 
particular Paced Tape session in the donor PSU vas 
transferred to the recipient PSU. The weights of 
students involved in the imputation were adjusted as 
follows: 

The students that remained in the donor PSU had 
their overall weight doubled by doubling the 
vithin-school student weight, since this one-half 
subsample also represented those students transferred 
to the recipient PSU. The overal\ weight for records 
in the recipient PSUs was the product of its original 
PSU weight, the other three weights, and the student 
nonresponse adjustment carried from the donor PSU. 
The weight associated with the allocation of the 
particular Paced Tape session, the doubled 
within-school weight, and th". student nonresponse 
adjustment were carried without modification from the 
donor PSU. The school weight of the donor school was 
adjusted by the ratio of the donor PSU weight to the 
recipient PSU weight, that is: 



ID 



2R 



. V 



2D 
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Post^stvatification 



The weights determined in the manner described 
in the preceding subsections were adjusted by 
post-stratif ication in order to reduce the sampling 
error of estimates relating to student populations 
that span several subgroups of the total population* 
Post^stratification replaced the "smoothing" that was 
done in the r^rior NAEP assessments. In the prior 
assessments, for each age class, a separate curve, 
one for each of six subclasses (e,g», bl?'*lcs in the 
South), was fitted by robust procedures to the total 
number of students as estimated by previous NAEP 
rounds to provide an adjustment target fipire to 
reduce discontinuities between successive NAEP 
assesom-^nts. For the 1983-8A assessment, smoothing 
was accomplished by post-ntratif ication, Tht 
post^stratification was performed for each grade/age 
group and separately for Spiral sessions, and for 
each of the four Paced Tape sesrions. It was 
performed by modifying the weights of the sample 
students in such a way that resulting e?=timates of 
the total nuiuoer of students in pecified rubgroups 
of the population were the same as the presumably 
better estimates that were made from other sources 
for each of these subgroups. (These procedures are 
described in detail in the Report Sample 
Selection, Weighting and Variance Estimation; 
NAEP-Year 15 issued by Vestat in 1985.) 

Weights Used for Variance Estimation (JKVTxx) 

Varia.ices for NAEP estimates are computed by a 
jackknife repeated replication method. This method 
invol"^s the use of a series of weights JKWTOl 
through JKVT32. 

For information about th^ use of these weights 
and how they were computed, see Chapter IV, Section 
H. 
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Data Collection, Scoring and Reporting 



IV. Data Collection, Scoring, and Reporting 



A. Sample Selection and Data Collection 

Westat was responsible for sample selection and field 
administration for the 1983-84 NAEP assessment (see Chapter III). The 
sample selection and data collection activities conducted by Westat 
included the following: 

1. Sample Selection (RTI/Westat) 

Based upon the 1983-84 sampling design, RTI selected and 
Westat invited more than 100,000 primary, elementary and 
secondary school students from more taan 1,400 schools in 64 
PSUs to participate in the 1983-84 assessment. Each 
administration Vc 3 coordinated by a district supervisor 
responsible for conducting sessions in four PSUs and for 
maintaining the security of NAEP materials and the 
confidentiality of assessment data. 

2. Field Administration (Westat, Inc.) 

Supervisors provided school personnel with general 
information about NAEP and worked with them to ensure student 
attendance at each assessment session. Each supervisor was 
assisted by two exercise administrators. Supervisors and 
exercise administrators were responsible for completing 
student sample selection in each school, establishing 
assessment schedules; ensuring that schools were properly 
prepared for sessions, collecting and checking all NAEP 
materials, and returning them to ETS. 

The weights provided on the Version 3.1 Public-Use Data Tapes were 
established through the .sampling procedures described above and in 
Chapter III. 

B. Data Entry System 

Due to the limited time available to ETS to organize the data 
collection activity for the 1983-84 assessment, it was not possible to 
design a scorable answer document. Rather, students marked their 
answers directly on assessment booklets. Therefore, it was essential 
to create a data entry system that would allow entry of information as 
it was received from the field. ETS designed an intelligent, direct 
data entry computer system to accommodate the BIB/UBIB design so that, 
when a booklet was presented to the system, it automatically prepared 
for appropriate exercise sets by setting up corresponding formats or 
data descriptions. The system also permitted on-line editing of the 
information entered into the computer. As a result, NAEP staff began 
processing each booklet upon ^receipt, and incoming data were analyzed 
much earlier than in previous assessments. 
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Data Editing 



Editing included an assessment of the internal logic and 
consistency of the data received* For example, data were examined for 
non-existent school IDs, out-of-range values, and illogical or 
inconsistent responses (e»g», a questionnaire in which a teacher 
indicated "I do not use a textbook for reading," but then entered the 
name of a textbook used for reading)* Where possible, conflicts in the 
data were resolved* If resolution was not possible, the information 
was left in the form in which it was received. 

1* Multiple Responses 

In cases where students, teachers, or school personnel 
provided more than one response to a single-tesponse item, 
specific guidelines and rules were developed to recode 
multiple responses in a consistent and accurate manner. For 
example, for Grade 4/Age 9 and Grade 8/Age 13, the text below 
appeared as item number 10 in exercise block J and as item 
number 26 in the common block in Paced Tape booklets: 

If you go home, who is usually there? 

A. No one 

B. My brother(s) and/or sister(s) 

C. My mother or father 

D. Another adult (Who?) 



In administering this item, NAEP personnel wanted to 
ascertain whether an adult was usually present when a student 
was at home. As a result, if a student responded to the item 
by circling B and C, the response was coded as C: circling B 
and D, the response was coded as D; circling C and D, the 
response was coded as C; or, circling A and D, the response 
was coded as D. All other combinations rere assigned a 
multiple response code. 

Professimally-Scored Items 

As in previous assessments, the 1983-84 session included items 
which required open-ended responses. In order to score these 
responses, NAEP recruited a team of readers, each of whom were trained 
in both holistic scoring (which concentrates on the overall quality of 
an essay) and in primary trait scoring (which focuses on specific 
characteristics of ^n essay). In both cases, readers used scoring 
guides specifically designed for each item. Some of the items were 
used in previous rt^ading/writing assessments, while others were 
developed for the 1983-84 session. The scoring guides for items from 
previous sessions ware reviewed and updated for the 1983-84 session, 
and the guides for newly-developed items were constructed using field 
tryout data. 

Twenty percent of the items were subjected to reliabili checks 
in which the responses were read by a second reader who was not 
provided with the score assigned by the first. Discrepancies in the 
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scores assigned by the first two readers were resolved by the scoring 
supervisor. The scores for open-ended items are included on the 
Public-Use Data Tapes (see the NAEP Technical Report for a detailed 
discussion of professional scoring). 

Quality Control 

NAEP measured the accuracy of its data entry operation to 
determine hov precisely the data moved from receipt of the instrument 
to the subsequent machine-readable dataset. For this purpose, a number 
of student booklets and questionnaires vere selected at random and 
compared, character by character, with their representation on a disk 
file. The number of booklets and questionnaires involved in quality 
control checks vas based upon the number needed to establish a 
statistically reassuring conclusion about the accuracy of the entire 
data entry operation. Student booklets and questionnaires for all 
three grade/ages vere included in quality control checks. 

Reading Item Analysis and Scale Derivation 

To produce NAEP reading results on a single scale, even though 
students responded to different sets of reading exercises, it was 
hypothesized that a single unobservable proficiency variable governed 
students' chances of responding correctly to a subset of 228 of the 252 
reading exercises. It vas also hypothesized that the same variable 
vould account for responses to the reading exercises in the Paced Tape 
administrations, possibly after a linear transformation to account for 
effects of the mode of administration. Using this unobservable 
proficiency variable, one can compare levels of performance across 
subpopulations or indicate trends over time as a single proficiency 
related to all items in the subset, rather than as percentages of 
correct response to individual items. The folloving sections describe 
the model employed to accomplish these objectives and relative 
variables found on the user tape. 

1. Item Response Theory 

Item response theory (IRT) models the probability that a 
given student vill response correctly to a given item as a 
function of a parameter characterizing the proficiency of that 
student and one or more parameters characterizing that item. 
Statements about students' proficiencies can therefore be made 
vith reference to a common proficiency scale even though 
students have responded to different items. This method 
facilitates comparisons of NAEP subpopulations, ages and 
grades, and time points, since analyses need not be limited to 
the examination of percents correct on items taken in common 
among comparison groups. 
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The particular IRT model employed for NAEP reading 
exercises was the three-parameter logistic model 



nx,. i|e,,a^,b.,c.) = c^ 

+ (1 - c.)/{l + exp[-l-7a^(e^ - b.)]} 

where 

x^^ is the response of student i to item j, 1 if 
correct and 0 if incorrect; 

6^ is the (unobservable) proficiency of si:udent i; 

a^ is the slope parameter of item j, characterizing 
its sensitivity to proficiency; 

b^ is the threshold parameter of item j, 
characterizing its difficulty; and 

c^ is the lower asymptote parameter of item j, 

reflecting the chance of a correct response from 
pupils of very low proficiency (c parameters were 
estimated for multiple-choice items, but fixed at 
zero for open-ended response items). 

Item Parameter Estimation 

The first step in implementing the IRT moael was the 
determination of the collection of items to which the model 
was to be fit. First, the assumption that all items would 
reflect a single proficiency led to concentration upon those 
items traditionally thought of as ^'reading exercises" 
(distinct from the NAEP "study skills" items that call for 
specific knowledge such as how to use tables of contents or 
bar charts). Second, only Spiral assessment booklets that had 
at least two eligible item blocks (those containing "reading 
items"~see Table IV. 1) were used to fit the model and to 
compute IRT item parameters. (Spiral assessment booklets that 
contained at least one of the eligible blocks were used to 
estimate reading proficiency scores — see Section IV. 4.) 

BIB/UBXB Spiral blocks and Paced Tapes containing items 
used for the IRT analysis are listed in Table IV. 1. BIB/UBIB 
Spiral booklets and Paced Tapes used for the IRT analysis are 
listed in Table IV. 2; the table indicates whe her the items 
contained in each booklet were used to estimate reading 
proficiency scores (S), were used to estimate reading 
proficiency scores and calibrate IRT parameters (C), or were 
not used in the IRT analysis at all (-). 
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Table IV. 1 



NAEP 1983-84 

BIB/UBIB Spiral Blocks and Paced Tapes Containing Items Used for IRT 



BIB/UBIB Spiral Blocks 





BIB/UB] 


[B Spiral 


L Block 




H 


J* 


K 


L 


M 


N 


0 


P 


Q 


R 


U 


V 


W 


X 


Y 


Grade 4/Age 9 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 






0 


Grade 8/Age 13 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 








X 


Grade 11/Age 17 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 


X 








X 



Paced Tapes 





Pacec 


I Tape 


P64 


P65 


P66 


P67* 


Age 9 


X 


X 


X 


X 


Age 13 


X 


X 


X 


X 


Age 17 


X 


X 


X 


X 



0 = block not administered to Grade 4/Age 9 students 

* Item N001801 (FLY) in Block J and Paced Tape 67 was not used to compute 
proficiency values 
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Table IV. 2 



NAEP 1983-84 BIB/UBIB Booklets and Paced Tapes 
Containing Items Used for IRT 



S = Items Used for Scoring Only 

C = Items Used for Scoring and Calibration 

- s Neither (booklet contains no usable reading items) 



Booklet 


Cr^dp 4/A&P 9 

\7L CIVIC ^/ *l|sC ^ 


Grade 8/Acre 13 


Grade ll/Af< 17 

x7 1. Civic X X r ^ 


1 


s 


s 


s 




c 


c 


c 


3 








4 








s 


s 


s 


s 


6 


s 


s 


s 


7 


c 


c 


c 


8 


c 


c 


c 


9 


c 


c 


c 


10 








11 


s 


s 


s 


12 


s 


s 


s 


15 


c 


c 


c 


14 


c 


c 


c 


15 


s 


s 


s 


16 


c 


c 


c 


17 


c 


c 


c 


IS 


s 


s 


s 


19 


c 


c 


c 


io 


s 


s 


s 


21 


s 


s 


s 


22 


c 


c 


c 


23 


c 


c 


c 


24 


s 


s 


s 


25 


c 


c 


c 


26 


s 


s 


s 


i1 


c 


c 


c 


is 


c 


c 


c 


29 


c 


c 


c 


30 


c 


c 


c 


ii 


c 


c 


c 


32 


c 


c 


c 


33 


c 


c 


c 


34 


s 


s 


s 


35 


c 


c 


c 


36 


s 


s 


s 


3^ 


s 


s 


s 


38 


c 


c 


c 


39 


s 


s 


s 


40 


s 


s 


s 
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Table IV. 2 
(continued) 



NAEP 1983-84 BIB/UBIB Booklets and Paced Tapes 
Containing Items Used for IRT 



S =: Items Used for Scoring Only 

C =: Items Used for Scoring and Calibration 

- =: Neither (booklet contains no usable reading items) 



Booklet 


Grade 4/Age 9 


Grade 8/Age 13 


Grade 11/Age 17 


41 


C 


C 


C 


42 


s 


s 


S 


43 


s 


s 


S 


44 








45 


c 


c 


C 


46 


s 


s 


S 


4? 


c 


c 


C 


48 


s 


s 


S 


49 


c 


c 


C 


50 


c 


c 


C 


51 


c 


c 


C 


52 








53 


c 


c 


C 


34 


s 


s 


S 


55 


c 


c 


C 


56 


c 


c 


C 


57 


c 


c 


C 


58 


c 


c 


C 


59 


c 


c 


C 


60 


s 


s 


S 


61 








62 


s 






63 


c 


s 


S 




Paced 
Tape 


Age 9 


Age 13 


Age 17 


P64 


C 


C 


C 


P65 


C 


C 


C 


P66 


C 


C 


C 


P67 


C 


C 


C 
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Across all grade/age levels in BIB/UBIB, 228 items were 
used in the IRT analysis. Appendix 1 (Chapter IX) lists each 
item used for the IRT analysis, with corresponding IRT 
parameters, standard errors and block or tape locations (block 
definitions are provided in Table VI,2 in Chapter VI). 

Performance for open-ended response items (essays) is 
coded by professional scorers using a four- or five-point 
scale (primary trait scoring) • For the IRT analysis, these 
ordered scales were dichotomized at the points shown in Table 
IV. 3. The IRT parameters are available for computer 
extraction from the Machine-Readable Catalog (see Table VI .3 
in Chapter VI). 

The parameters of the items were then estimated using a 
prototype of an updated version of the BILOG computer program 
(Mislevy and Bock, 1982), modified to allow for items with 
different numbers of response alternatives. BIB Spiral 
responses from all grade^age groups were included in a single 
run. Only students who were administered a booklet containing 
at least two of the item blocks eligible for IRT analysis 
(Table IV. 1) were included in the item parameter estimation 
run. However, once the items parameters were estimated, 
reading proficiency plausible values (see Section 4 below) 
were estimated for any student who was administered a booklet 
containing one or more of the eligible item blocks. The scale 
of the item parameter estimation run was set by standardizing 
the combined distribution of all students from all grade/ages 
responding to the booklets listed in Table IV. 2. The numbers 
of students used for IRT parameter estimation were 13,913 for 
Grade 4/Age 9; 13,472 for Grade 8/Age 13; and 13,759 for Grade 
11/Age 17. 

Paced Tape items were estimated separately for each age, 
along with items from past reading assessments for that age. 
These Paced Tape scales were linked to the BIB Spiral scale by 
transforming the Paced Tape parameters so as to match the 
first two moments of the age population, as estimated from the 
Paced Tape sample, to the corresponding moments estimated from 
the randomly equivalent BIB Spiral sample. (See the NAEP 
Technical Report for details.) Resulting item parameter 
estimates appear in Appendix 1, accompanied by standard errors 
of estimation for each parameter. 

The Reading Proficiency Scale 

The 9 scale, while theoretically unbounded, spans an 
effective range of about -4 to +4 in the NAEP populations. To 
facilitate discussion, an alternative "reading proficiency" 
(RP) scale was also derived. The RP scale is a number-correct 
score on a hypothetical 500-item test, comprised of items with 
equal "a" values of 1.5, equal "c" values of 0, and 
equally-spaced "b" values from -5.00 to +4.98. An RP value 
corresponding to a given 9 value is the expected score on this 
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Table IV. 3 



NAEP 1983-84 

Points of Dichotomization of Open-Ended Response Items 

Used for IRT 





Point of 


Block 


Item 


Dichotomization* 


Gr. 4-Age 9 


Gr. 8-Age 13 


Gr. 11-Age 17 


N001509 


5 


H 


H 


n 


N001906 


3 




J 


J 


N002806 


3 


L 


L 


L 


N003106 


3 


M 


M 


M 


N003706 


2 


N 


N 


N 


N004305 


2 




0 


0 


N004607 


2 




P 


P 


N007410 


3 




u 


u 


N007506 


3 




V 


V 


NC08210 


5 




Y** 


Y 


N008907 


2 


J 






N015907 


2 






Q 



* Ratings lover than the dichotomization point were scored as "incorrect"; 
ratings higher than or equal to the dichotomization point were scored as 
"correct . " 



** Item N008210 also appeared in Paced Tape 66 for Grade 8/Age 13. 
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hypothetical test for a student with proficiency 9. The 
relationship is virtually linear in the region of 9s from -4 
to +4, and is approximated by 



RP = 509 = 250.5 



Plausible Values 

In most applications of IRT, precise information about 
each sampled student is desired for purposes of individual 
diagnosis, selection or placement* Typically, each student is 
administered a sufficient number of items to ensure an 
accurate estimate of proficiency; testing times of several 
hours are not unusual* 

More efficient estimates of the distribution of 
proficiencies in a group of students can be obtained using 
NAEP's BIB Spiral sampling design, which solicits only a few 
responses per sampled student. 

The advantages of more efficient estimation of population 
characteristics are countered by the inability to make precise 
statements about individuals. Point estimates of 9 or RP that 
are in some sense optimal for each sampled student can lead to 
seriously biased estimates of population characteristics. One 
can, however, express what is known about student i's value, 
once his or her item responses (x^) and background and 
attitude responses (y^) have been observed, in terms of a 
plausible distribution for the students' 9: P(9|x^,y.)* The 
value of any function t of reading profic ency and^ot?ier 
background variables could then be estimated from the NAEP 
data by evaluating the integral 



£[t(9,y)Ix,jrl = I t(9,j^) p(9|x,jr) d9 

where vector-valued variables represent values over all 
sampled students. 

Because evaluating such integrals is generally difficult, 
the user tapes provide partial computations to facilitate user 
applications. For each sampled student who was administered 
at least one of the IRT item blocks, five values are provided, 
selected at random from p(9|x^,y^), where x^ denotes the 
student's responses to all of the 228 calibrated items to 
which he or she has responded, and y^ denotes status on age, 
grade, sex, race/ethnicity, size and type of community, 
parental education, and region of the country. These values 
may be referred to as "plausible values" of 9 for student i. 
Corresponding values on the RP scale are also provided. 
Justification, details and properties of plausible values are 
provided in the NAEP Technical Report. The number of students 
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for vhich the plausible values are provided on the BIB/UBIB 
tape files are 23,132 for Grsde 4/Age 9; 23,665 for Grade 
8/Age 13} and 24,055 for Grade 11/Age 17. 

To estimate a given function t involving 9 or RP, one 
evaluates t using the n* (n « 1, 2, 3, 4, or 5) plausible 
value for each student involved, as if those values were the 
true (and unobservable) 9 or RP. The result is a good 
2stimate of t when t involves only proficiency and one or more 
of the background variables specified above. Biases may 
result when other background variables are involved (see the 
NAEP Technical Report for details). Computing t with a 
different set of plausible values leads to a different but 
equally good estimate of t. The best estimate currently 
afforded is the average of all l^.ve such estimates, one 
corresponding to each set of plausible values. The variance 
among the five estimates reflects uncertainty due to not 
knowing precise values of 9 or RP, and should be added to the 
sampling variance. (See Section G below concerning how to 
compute sampling variances, and Chapter V concerning ho^ to 
combine variance estimates.) 

Average Response Method (ARM) of Writing Proficiency 

Standard item response theory has been developed to estimate an 
ability level based on a student's response to a pool of items, each of 
which has only two possible scores: acceptable or unacceptable. In 
contrast, the items for the assessment of writing proficiency require 
open-ended response, that is, essays. NAEP essays were scored by 
readers who evaluated specific characteristics of th2 essays (primary 
trait scoring). 

Each essay was assigned a score on a five-point scale, defined as 
follows: 

0 = no response (but exposed to the item) 

1 = unsatisfactory response 

2 = minimally satisfactory response 

3 = satisfactory response 

4 = elaborated response 

The use of primary trait scoring to rate writing items precludes 
the use of standard IRT technology to model writing proficiency. While 
a number of generalizations of IRT procedures have been proposed to 
account for the characteristics of such types of ratings, their 
applications to these data have not yet yielded productive results. 

It is, however, still desirable to be able to provide a common 
measure of writing ability to estimate the performance of any group or 
subgroup, even though no member of any subgroup took all of the items 
in the writing item pool. To accomplish this, the measure of writing 
ability has been defined to be the predicted mean score across a set of 
10 writing items. This measure of proficiency is derived by a 
technique called the Average Response Method (ARM), which is a kind of 
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multiple regression* The ARM writing proficiency value for a given 
student is a prediction of what that student's mean score across the 10 
writing iteiss might plausible be, based on the student's scores on the 
writing items taken and the student's status on the background 
variables of grade, sex, race/ethnicity, size and type of community, 
parental education and region of the country* 

The writing items selected to constitute the ARM writing value and 
the blocks containing them, by grade, are shown in Table IV. 4. These 
items constitute the complete set of writing items which were (1) part 
of the BIB Spiral Dooklets and (2) presented to at least two grades* 
(Item N000502, which was given only to Grade 8, was included because it 
could be linked to item N000602, which was given in two grades*) At 
present, ARM values are provided only for students in the modal gra3e 
(Grad^ 4, B or 11) who responded to at least one of the 10 writing 
items * 

!• The Basic Model 

The assumed model for predicting a given student's ARM 
writing proficiency value, based on the student' bc'ckground 
characte "istics and scores on m of the 10 writing items, is 
the linear model 



y^=A.r=X0=Y. 

-'ik 1 1^ 'ik 

where 

y^^ is the k^^ plausible value (in the sense of 

Section F.4 above) of the ARM proficiency value 
for the i student; 

A^ is the 1 X k vector of constants giving the 



particular student's status on the various 
background variables; 

is the k X 1 vector of the effects of the 
oackground variables; 



is the 1 X m vector of scores received by the i 
student on the m writing items actually taken; 



. th 



^Ve intend to estimate ARM v?lues for students of the modal age 
(9, 13 or 17) but not in the modal grade — this will allow analyses by 
age* Details will be provided as an update* 



Table IV. 4 



NAEP 1983-84 

Writing Items Used for the ABM Writing Proficiency 





Block 


Item 


brace 4 


Grade o 


Grade 11 


N000102 


A 


A 

A 


A 


N000202 


D 

o 


D 
D 


n 
D 


N000302 




C 


c 


N000402 


D 


D 


D 


N000502 




E 




N000602 


E 


E 




N000702 


F 


F 


F 


N000802 


F 


F 




N000902 


G 


G 




N001002 


G 


G 


G 



-SO- 
BS 
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P is the m X 1 vector giving the change in 

proficiency value for a unit change in the scores 
on each of the m items; and 

r^^ is an estimated residual draw from the predictive 
distribution of the ARM value given the observed 
values of and (discussed below). 

Item Parameter Estimation 

The parameters relating performance on a given set of 

writing items O) and background characteristics (T) with 
mean performance on the complete set of 10 items were 
estimated by least-squares technology. To accomplish this it 
is sufficient to obtain estimates of the means, variances and 
inter-item covariances, by demographic subgroups, for the 
complete set of writing items. Because the ARM writing 
proficiency value is the mean of the 10 items, this in turn 
produces estimates, by demographic subgroups, of the 
proficiency value mean and variance as well as the covariances 
between the proficiency score and each of the 10 writing 
items. These provide a complete set of sufficient statistics 
for the standard least-squares prediction of an ARM 
proficiency value given background characteristics and scores 
on any subset of the 10 writing items. 

Although no student was presented with ^lore than 4 of the 
writing items and the majority of students were presented with 
only 1 or 2, the NAEP BIB Spiral design assured that every 
writing item assigned to be assessed in a given grade was 
presented to a representative sample of students from that 
grade. Furthermore, the sample of students in a given grade 
who were presented with any pair of the writing items also 
constituted a representative sample of students from that 
grade. This means that unbiased and consistent estimates of 
item means, variances, and inter-item covariances for the pool 
of writing items assigned to a given grade can be derived from 
the information provided by the respondents to those items. 
This allows an estimate of the normal equations that would 
have been obtained had all students in the grade been 
presented with all items that were assigned to that grade. 
Details of the procedure used to estimate these normal 
equations based on the sample estimates of the means, 
variances and covariances appear in the NAEP Technical Report. 

Because not all items were presented at Grades 4 and 11, 
the normal equations for those grades has missing cells, 
corresponding to the items which were not presented. For 
Grade 4, there were 2 missing items (N000302 and N000502). 
The rells of the normal equations corresponding to these 



-51- 



missing items (which includes all sums of crossproducts 
involving either missing item) were completed in the following 
manner: 

1) It was assumed that, for the population of Grade 
4 students, the conditional distribution of the 
two items given the background rharacteristics 
and responses to the 8 items actjp.lly assigned to 
Grade 4 is the same as the equivalent conditional 
distribution for the population of Grade 8 
students, and is multivariate normal. 

2) This conditional distribution was estimrted for 
the Grade 8 sample. 

3) This estimate was combined with the marginal 
distribution obtained from the Grade 4 normal 
equations to obtain an estimate of the joint 
distribution of all items for the population of 
Grade 4 students. 

Under the assumption in (1) above, this will produce an 
estimate of the normal equations that would have been obtained 
had all items (including the missing 2) been presented to each 
member of the population of Grade 4 students. A similar 
procedure was used to estimate the full set of normal 
equations for Grade 11. 

These three sets of normal equations were then pooled and 
used as the basis for the least-squares prediction of a give*: 
student's ARM writing proficiency value based on that 
student's background charactf^ristics -^nd responses to the 
writing items actually taken. 

3. Plausible Values for the ARM Writing Value 

The basic predicted value for a given student is, in the 
above notation, 



Y = A.r = X.3 
1 1 1 *^ 

Because of the method of estimation employed, this basic 
predicted value is an optimal (in a least-squared sense) point 
estimate of the student's score. Additionally, optimal point 
estimates of average scores for the various background 
characteristics included in the background variables in the 
fit can be obtained by averaging the basic predicted values 
across the appropriate subgroup. 

However, as is the case with the Reading Proficiency 
Scale values, these optimal point estimates can lead to 
seriously biased estimates of the characteristics of 
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populations not addressed by the background variables^ 
Additionally, the variability of any of the estimates 
(including those for populations addressed by the background 
variables) will be seriously understated by using the basic 
predicted values* 

The basic predicted scores do not take into account the 
distribution of potential scores for any individual. The 
basic predicted value for a student is an estimate of the mean 
of this distribution and, as such, makes no statement about 
the set of other values, any one of which might also have been 
the student's score. 



As in the case of reading proficiency, the value of any 
function of writing proficiency should be obtained by 
evaluating the expectation ot the function across the joint 
distribution of the parameter estimates and the potential 
scores for each student, partial computations are provided on 
the user tapes to facilitate user applications by eliminating 
the need to evaluate integrals explicitly. The user tapes 
provide five "plausible values" of writing proficiency for 
each Scudent. The i^ plausible value of a student is the sum 
of: 

1) the basic predicted value for the student; and 

2) a random term reflecting the uncertainty of the 
student's value given the student's background 
variables and scores on items taken and holding 

the parameters O and T) fixed. 

These plausible values should be used in the same m-^nner 
as indicated in Section F.4 above. As in the case of reading 
proficiency values, statistics involving writing proficiency 
and background variables not specified above are subject to 
biases. Further details on the use of plausible values are 
provided in Chapter V. 

H. Calculating Standard Errors (Jackkniting) 

NAEP uses a complex, multi-stage sample design. Most statistical 
dnalysis programs assume simple random sampling. This assumption, 
given the NAEP design, would produce standard error estimates ihat are 
systemat really too small. (Some of the reasons for this are discussed 
in Chapter V. ) 

To estimate sampling variability properly, the characteristics of 
the sample design must be taken into account. One way to obtain a 
proper estimate of sampling variability is through the jackknife 
method, a particular version of which is used for NAEP standard ecror 
estimation. 
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The jackknife vari?.nce estimation procedure can be applied to 
variance estimation for both common statistics^ such as means or 
proportion-correct^ and more complex statistics, such as regression 
coefficients. 

The jackknife procedure estimates the sampling variability of a 
statistic t by measuring how much the value of that statistic changes 
when carefully selected subsets of the data are changed. 

An account of the statistical properties of the jackknife method 
used by ETS appears in the NAEP Technical Report. Additional details 
on certain properties of estimates of sampling variability can be found 
in Chapter V of this guide. 

This section provides the operational details for the jackknife 
staiidard error estimate as it is used for the NAEP data analysis. Two 
techniques are described below. The first cechnique, the multi-weight 
method, was used in the production of the NAEP report, The Writing 
Report Card; Writing Achievement in American Schools, 1984 (1986). 
The second technique, the single-weight method, is a somewhat less 
accurate approximation which is, in some cases, easier to compute. The 
single-weight method was used in the production of the NAEP reports. 
The Reading Report Card: Trends in Reading over Four National 
Assessments 1971-1984 (1985) and NAEP; Writing Trends Across the 
Decade, 1974- 1 984 (1986). 

The jackknife procedure involves identifying pairs of Primary 
Sampling Units (PSUs). This pairing takos the sample design into 
account. In the 1983-84 assessment, where 64 PSUs were selected, 
pairing results in 32 PSU pairs. Within a pair, each PSU was randomly 
designated as the first or the second member of Ihe pair. 

The jackknife method estimates the sampling variability of any 
statistic as the sum of components of variability which may be 
attributed to each of the PSU pairs. The variance attributed to a 
particular PSU pair is measured by estimating how much the value of the 
statistic would change if the information embodied in the PSU pair were 
to be changed. 

This estimation is done by the computation of a quantity t called 
a pseudo-replicate, which is associated with the i^^ PSU pair, and 
which is a certain estimate of the statistic of interest t. 
Specif iCiXlly, the i pseudo-replicate of the statistic t is created by 
eliminating the data from the first PSU of the pair, replacing the lost 
information with that from the second PSU of the pair (so that the 
second PSU is include twice), and re-estimating the statistic based on 
this altered set of data. 

The jackknife estimate of the variability of the statistic t used 
by NAEP is 
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32 



Var(t) 



and the estimated standard error is the square root of this value. 
Note that there are a variety of alternative jackknife estimates of 
variance available in addition to the one given here. For a discussion 
of the properties of these various estimates, see the NAEP Technical 
Report. 

In practical terms, the major expenditure of resources in the 
computation of a jackknife variance estimate occurs in the construction 
of the pseudo-replicates. Two methods are presented to construct the 
pseudo-replicates. The first method, which is the method used by NAEP, 
is called the multi-weight method. This method is applicable to the 
estimation of a wide range of statistics, is straightforward in its 
implementation, and more properly accounts for sources of variability 
due to nonresponse adjustment and post-stratification. Implementation 
of this method requires 33 passes through the data. 

The second method, the single-weight method, requires only one 
pass through the data but requires special programming and 
approximately twice the computer storage space of the multi-weight 
method. The single-weight method does not account directly for the* 
variability due to post-stratification. 

1. The Multi-Weight Method 

The multi-weight method can be applied in the estimation 
of the sampling variability of any statistic which can be 
expressed as a function t(;/,w) of sample responses ^ and 
weights w. For example, t 'can be a weighted mean, a weighted 
proportion, a weighted percentile point or a more complex 
statistic, such as a weighted regression coefficient (or 
vector of regression coefficients). The computation of 
pseudo-replicates of any such statistic by the multi-weight 
method involves the use of 32 sets of weights, referred to as 
JKWTOl through JKVT32 on the user tapes. The set of weights 
JKWTi is identified with the i^ PSU pair and is used to 
compute t^, the i*^^ pseudo-replicate of the statistic t. The 
value of this pseudo-replicate is 



t^ = t(y, JKWT i) 

which is simply the statistic t recomputed by using the 
weights JKWTi instead of the sample weights (WEIGHT). 
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The set of weights for the i^^ PSU pair, JKWTi, are 
computed as follows : 

1. Let be the base weight for student j. The base weight 

is the full student weight (WEIGHT) without the 
adjustments for nonresponse and post-stratification, 

2. Let 



0 if student j is in the first PSU of 
PSU pair i; 

2W^ if student j is in the second PSU of 
^ PSU pair i; and 

if student j is not in either PSU of 
PSU pair i. 

This set of pseudo-replicate base weights effects the 
elimination of the first PSU of the pair and replaces it 
in the sample with the second PSU of the pair. 

3. Adjust the set pseudo-replicate base weights produced by 
Step 2 for nonresponse and post-stratification by treating 
them as if they were base weights for the sample. These 
adjustments take into account the grade/age of the student 
and the mode of administration. 



j 



Because JKWTi is the set of pseudo-replicate base weights 
after adjustment for nonresponse and post-stratification, the 
effects of those adjustments on the value of the statistic t 
are approximately accounted for in the estimate of the 
variance of t attributed to the i*^^ PSU pair. 

From the user's standpoint, it is straightforward to 
obtain the jackJcnife variance of any statistic by the 
multi-weight method. Furthermore, no special programming is 
required to compute the pseudo-values regardless of the 
stetistic. All that is required is 33 passes through the 
data: the first pass tc compute t'le overall statistic t using 
the student weight (WEIGHT) and 32 passes to compute the 
pseudo-replicates t^ using the jackknife weight (JKWTi). By 
appropriately accumulating intermediate results for each of 
the 33 weights, it is possible to compute the necessary 
statistics in one pass through the data. However, separate 
computations for each weight and some special programming are 
still required. 

As an example, details for the computation of the 
jackknife variance of a weighted mean follow (SPSS-X and SAS 
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code for this purpose are contained in Table VI, 10 of Chapter 
VI.) 

Let be the value of some measurement of interest for 
student k and let W. be that student's full-student weight 
(WEIGHT). The statistic of interest is the weighted mean 
value of Z: 



Note that if Z^^ can only take the values 0 or 1, then t 
is the weighted proportion receiving a vcxlue of 1. 

Let W^^ = value of JKWTi fcr student k* The 

k 

pseudo-replicate for the i^^ PSU pair is 

n n 

t = S W^Z^/ Z Ml 
k k k 

k=X k«l 



The jackknife variance of the weighted mean is 



32 

Var(t) = Z (t. - t)' 

1 = 1 

and the jackknife standard error of the mean is the square 
root of Var(t). 

The Single-Veight Method 

The multi-weight method requirement of 33 separate sets 
of computations makes it a computationally expensive 
procedure. An alternative jackknife method, the single-weight 
method, requires only one pass through the data, decreasing 
costs through fewer computations. However, the single-weight 
method requires special programming for each genera] statistic 
and about two times the computer memory required for the 
.nulti-weight method. Furthermore, this method does not 
directly account for variability attributable to 
post -St ratification adjustments. 

In the single-weight method, a student's pseudo-replicate 
weight is the same for all pseudo-replicates except the 
pstudo-replicate corresponding to the PSU pair containing the 
student. This frequently allows the pseudo-replicates and 
overall estimate to bi expressed as functions of various PSU 
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level totals, where these totals can be accumulated in a 
single pass through the data. 



Specifically, let 



W. . = full-student weight (VEIGHT) for the k*"*" student 
of the j^^ PSU of the i^^ PSU pair. 

The pseudo-replicate weight for this student corresponding to 
the h^^ PSU pair is 



ijk 



0 h = i, j 

2W, h = i, j 



1; 

2. 



That is, the student's pseudo-replicate weights, for all PSU 
pairs but the pair that contains the student, are identically 
equal to the full-student weight. For the PSU pair containing 
the student, the student's pseudo-replicate weight is either 0 
or twice his or her full-student weight depending on which PSU 
of the pair contains the student* 

Ihese pseudo-replicate weights (which are different from 
the JKWTi) allow the pseudo-replicates of a weighted mean to 
be expressed as ratios of sums and differences of PSU totals* 
(SPSS-X and SAS code for this case are contained in Table 
VI. 12 of Chapter VI.) 



Let Z. . be the student's value of interest. 

1 3 K 



Define 



ni j 

13+ 13k 
k = l 

and 



ni3 

13+ i3ki3k ' 

k = l 

where nij is the number of students in the j*^** PSU of the i*^ 
PSU pair. 

The overall mean and all pseudo-values can be obtained 
from these 64 PSU totals. 
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Let 



W 



+++ 




and 




The overall mean is t = X 
pseudo-replicate is 



The i 



th 



+ + + 



+ + + 



The single-weight variance estimate is 



32 



Var*(t) = E (t* - t) 



2 



i = l 



Notes: 



(1) The single-weight variance Var*(t) is not equal to 
the multi-weight variance Var(t), although they generally will 
be close* 

(2) The formulas for the 'tingle-weight pseudo-replicates 
of any other type of statistic mjst be determined on a 
case-by-case basis. 



I. Reporting Subgroups 

NAEP reporLs performance results for groups of students rather 
than for individual students. In addition to reporting national 
results, NAEP reports information about student subgroups defined by 
sex, race/ethnicity (both observed and imputed), region of the country, 
grade/age, level of parent's education, and size and type of community. 

Some subgroup data were not obtained directly from assessment 
responses, but were derived through procedures described in Sections 
3, 6 and 7 below. 



Subgroup data are contained under the variable names listed in 
Table IV. 5. 
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Table IV. 5 
NAEP 1983-84 
Reporting Subgroup Variables 



Subgroup 


Variable Name 


*?tiidpnt Pi 1 p 




Sex 


SEX 




Observed Race/Ethnicity 


RACE 




Imputed Race/Ethnicity 


ETHNIC* 




Region 


REGION 


SREGION 


Age 


STUDAGE* 




Grade 


NEWGRD 




Size & Type of Community 


STOC 


SSTOC 


Parent's Education 


PARED* 





* Denotes derived variable 



The reporting subgroups were determined as follows: 
1* Sex 

Responses were reported for male and female students. 

2. Observed Race/Ethnicity 

This is the observed race/ethnicity of the student being 
assessed. The observed definition of *. ludent race/ethnicity 
was the only one used in NAEP assessments prior to 1983-84. 
This variable should be used for race/ethnicity subgroup 
comparisons to previous assessments. 

3. Imputed R a ce/Ethnicity 

This is an imputed definition of race/ethnicity, derived 
from several sources of information. This variable can be 
used for race/ethnicity subgroup comparisons within the 
1983-84 assessi'ient. 

Three common background items were used to determine 
race/ethnicity for -rudents who participated in the 1983-84 
assessment session. The items were included in every Spiral 
assessment booklet and in each Paced Tape booklet, as follows: 
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Common Background Item Number 2; 
2. Are you Hispanic? 

A. No 

B. Yes, Mexican, Mexican American, or Chicano 
Yes, Puerto Rican 

D» Yes, Cuban 

E. Yes, Other Spanish/Hispanic 
(What?) 



Students vlio responded to item number 2 by circling B, C, 
D, or E were considered Hispanic. For students who circled A, 
did not respond to the item, or provided information which was 
illegible or which could not be classified, responses to item 
number 1 were examined in am effort to determine 
race/ethnicity. Item number 1 read as follows: 

Common Background Item Number 1: 
Tl Are you: 

A* American Indian or Alaskan Native 

B* Asian or Pacific Islander 

C. Black 

D. White 

E. Other (What?) 



Students who circled A were considered American Indian; B 
were considered Asian; C were considered Black; and D were 
considered White. If a student responded by circling E, 
race/ethnicity was determined in accordance with the 
information filled in by the student as "Other (What?)." 

For students who did not respond to item number 1, or who 
did so by providing illegible information or information which 
could not be classified, responses to item number 4 were 
examined in an effort to determine race/ethnicity. Item 
number 4 read as follows: 

Common Background Item Number 4: 

4. What language do most people in your home speak? 

A. English 

B. Spanish 

C. Another language 

(What is it?) 



A student was considered Hispanic if he or she circled B. 
For a student who circled C and indicated that most people in 
the home spoke languages which were not English or 
Spanish/Hispanic, race/ethnicity was determined by classifying 
the language specified by the student. 

For a student vho did not respond to common background 
items 1, 2 or 4 above, observed race/ethnicity, if provided by 
the exercise administrator, was used- 
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Race/ethnicity could not be classified for a student who 
did not respond to background items 1, 2 or 4, and for whom an 
observed race/echnicity was not provided ♦ 

The races and ethnicities which were provided by students 
in response to items 1, 2 and 4 above are listed in Table 
IV. 6. Slashes indicate variations in the way races and 
ethnicities were sp'^lled by students. 

Table IV. 7 summarizes the procedure used to determine 
race/ethnicity. 

Size and Type of Communit y 

NAEP assigned each participating school to one of seven 
Size and Type of Community (STOC) categories. The categories 
were designed to provide information about the communities in 
which the schools were located. 

The STOC reporting categories consist of three "extreme" 
types of communities and four "residual" community sizes. 
Schools were placed into STOC categories based upon 
information about the type of community, the size of its 
population and upon an occupational profile of residents 
provided by school principals. The principals completed 
estimates of the percentage of students whose parents fit into 
each of six occupational categories. 

Schools in extreme rural and low or high metropolitan 
areas were ranked in descending order according to the 
occupational profile, the type of community, and the size of 
its population. The top 10 percent of these schools were 
assigned to the extreme STOC categories (1, ? and 3) below. 
The remaining schools were classified according to one of the 
four residual STOC categories. The three extreme STOC 
categories are as follows: 

STOC 1 - Extreme Rural : 

This category was used for schools in rural areas where a 
high proportion of adults were farmers or farm workers and a 
low proportion of professional, managerial, or factory 
workers. At least some of the students in these schools were 
from open country or places with a population of less than 
10,000. 

STOC 2 - Low Metro : 

The low metro STOC category wai used for schools in areas 
where a high proportior of the adult population was either not 
regularly employed cr on welfare and a low proportion was 
employed in professional or managerial positions. The schools 
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Table IV. 6 
Race/Ethnicity Classifications 



American Indian ; 
American Indian 
Cherokee 
Indianamerican/ 
Indiamerican 
Nativeam 
Navahoe 
Sueinda 



Black ; 

Af roamer i can 
Black 

Blackamericrin 



Hispanic ; 
Chicano 
Columbian 
Dominican 
Hispanic 
Latin 
Mexican 
Puerto Rican 
Salvadorian 
Spanish 



White ; 

Appropriate rac:: 
White. Races/et 
sidered American 
or White were in 



Asian; 

Amerasian/Amasian 

Americanphillippine 

Asian 

Assirian 

Cambodian 

Chinese 

Eastindi 

Eurasian 

Filippine/Fillippine/ 

Filappine/ 

Phillipine 
Fr India 
Guamania 
India 

Indianasian 

Indonesian 

Japanese 

Japanes e-Amer i can 
Korean 

Lasos/Leocean/Laot ion/ 

Loas 
Oriental 
Pacific 
Pakistan 

Taiwanes/Taowames 
Thai/Thia 

Vietnamese/Vei tnamese 



Unclassified : 
s/ethnicities were classified as 
hnicities which could not be con- 
Indian, Asian, Black, Hispanic 
eluded as unclassified. 
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Table IV* 7 

NAEP 1983-84: Determining Race/Ethnicity 



Background Item Number 2 
2. Are you Hispanic? 
A. No 

B* Yes, Mexican, Mexican American 

or Chicano 
C* Yes, Puerto Rican 
D* Yes, Cuban 

£♦ Yes, Other Spanish/Hispanic 
(What?) 



student circled , did not respond, 
provided •ithttr an ill^gibl* rvspons* 
or A r«8pon8« which could not be 

classified 



Background Item Number 1 
1* Are you: 

A* American Indian or Alaskan Native 
B* Asian or Pacific Islander 

Black 
D- White 



student did not circle h , B , Z or D; 
provided either an illegible response 
or A response which could not be 

cla&dified 



Background Item Number 1 
E. Other (What?) 



student did not circle E, provided 
either an illegible response or a 
response which could not be classified 



Background Item Number 4 
What language do most people in your 
home speak? 
A* English 
B* Spanish 
C* Another language 
(What is it?) 



student circled A, did not respond, 
provided either an illegible response 
or a response which could not be 

clas^2 fiea 



Observed R; c^/Ethnicity 



Observed Race/Ethnicity was not 
provided by Exercise Administrator 



Student circled 

B , C , D or E 



Unclassified Race/Ethnicity 



student circled 
A , B , C or D 



Student filled-in another 
Race/Ethnicity 



Student 
was 

Hispanic 



Istudent was: 
A. American 

Indian, 
B* Adian, 

C. Black, or 

D. White 



Student was: 
American 
Indian, 
Asian, 
Black, 
White; or 
Hispanic 



student circled B or circled C 
and filled-in a language — 



Student was: 


B* 


Hispanic, 


C. 


American 




Indian, 




Asian, 




Hispanic, 




Black, or 




White 



Provided by 

Exercise Administrator 



Student was: 
American 
Indian, 
Asian, 
Black, 

Hispanic, or 
White 
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in STOC 2 were located in cities, or the urbanized area of 
cities, with a population greater thar 200,000. 

STOC 3 - High Metro : 

High metro schools were located in city areas where a 
high proportion of adults was employed in professional or 
managerial positions and a low proportion factory or farm 
workers, not regularly employed, or on welfare. STOC 3 
schools were located in cities or the urbanized area of cities 
with populations greater than 200,000. 

Schools which did not fall into STOC 1, 2 or 3 were 
classified according to four "residual" STOC categories 
depending upon the size of the community in which they were 
located. The four residual STOC reporting categories are as 
follows: 

STOC 4 - Main Big City : 

^lOC 4 schools were located within the limits of cities 
with populations greater than 200,000 but not classified as 
High or Low Metro. 

STOC 5 - Urban Fringe : 

The schools assigned to STOC 5 were located in the 
urbanized area, but outside the limits, of cities with 
populations over 200,000, but not classified as Low or High 
Metro. 

STOC 6 - Medium City : 

STOC 6 schools were located in cities with populations of 
between 25,000 and 200,000 which did not classify as fringe 
areas for big cities. 

STOC 7 - Small Place : 

The schools assigned to STOC 7 were located in 
communities with populations of less than 25,000. These 
communities were not located in Lhe urbanized areas of big 
cities and could not be classified as Extreme Rural. 



5. Region 

In addition to overall responses, NAEP computed data for 
four geographical regions in the United States. Table IV. 8 
outlines the assignment of individua"' states to each region. 
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Table IV. 8 
Geographic Regions 



NORTHEAST: 


SOUTHEAST: 


Connecticut 


New Hampshire 


Alabama 


Mississippi 


Delaware 


New Jersey 


Arkansas 


North Carolina 


District 


New York 


Florida 


South Carolina 


of Columbia 


Pennsylvania 


Georgia 


Tennessee 


Maine 


Rhode Island 


Kentucky 


Virginia 


Maryland 


Vermont 


Louisiana 


West Virginia 


Massachusetts 






CENTRAL: 


WEST: 


Illinois 


Missouri 


Alaska 


New Mexico 


Indiana 


Nebraska 


Arizona 


Oklahoma 


lova 


North Dakota 


California 


Oregon 


Kansas 


Ohio 


Colorado 


Texas 


Michigan 


South Dakota 


Hawaii 


Utah 


Minnesota 


Wisconsin 


Idal.o 


Washington 






Montana 


Wyoming 






Nevada 





6. Parental Education 

Students were asked to indicate the extent of their 
father's education in one of the following ways: 

1) He did not finish high school; 

2) He graduated from high school; 

3) He wsnt to another school after graduating 
high school; 

4) He graduated from college; or 

5) I Don't Know. 

Students were asked to provi"* . the same information about 
the extent of their mother's education by checking one of the 
following: 

1) She did not finish high school; 

2) She graduated from high school; 

3) She went to another school after graduating 
high school; ^ 

She graduated from college ^ or 
5) I Don't Know. * 

The information was combined into one parental education 
reporting category, as follows: 

If a student indicated the extent of education for either 
parent, che higher of the two levels was included in the data. 
If a student indicated that he or she did not know the level 
of education for both parents or indicated that he or she did 
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not know the leve3. of education for one parent and did not 
respond for the other, the parental education level was 
classified as unknown* If the student did not respond for 
either parent, the student was recorded as providing no 
response* 

Grade/Age 

In an effort to enhance the utility of assessment data, 
NAEP began sampling students by grade as well as age during 
the 1983-84 assessment- As a result, 1983-84 data reflect the 
following grade/age classifications: 

Grade 4/Age 9 Stu r*.- ts; 

For the Grade 4/Age 9 sample, age was computed 
as of December 31, 1983. The sample includes many 
students who were both in grade 4 and age 9. 
However, because NAEP collected data by grade or age 
during the 1983-84 assessment, the Grade 4/Age 9 
sample also includes students who were age 9 (born in 
1974) but not in grade 4, students who were in grade 
4 but age 8 or younger (born in or after 1975), and 
students who were in grade 4 but age 10 or older 
(born in oi before 1973). 

Grade 8/A g e 13 Students ; 

For the Grade 8/Age 13 sample, age was computed 
as of December 31, 1983. The sample includes many 
students who were both in grade 8 and age 13* 
However, because NAEP collected data by grade or age 
during the 1983-84 assessment, the Grade 8/Age 13 
sample also includes students who were age 13 (born 
in 1970) but not in grade 8, students who were in 
grade 8 but age 12 or younger (born in or after 
1971), and students who were in grade 8 but age 14 or 
older (born in or before 1969). 

G rade 11/Age 17 Students : 

For th i Grade 11/Age 17 sample, age was computed 
as of September 30, 1984. Th(i sample includes many 
students who were both in grade 11 and age 17. 
However, because NAEP collectea data by grade or age 
during the 1983-84 assessment, the v^rade 11/Age 17 
sample also includes students who were age 17 (born 
between October 1, 1966 and September 30, 1967) but 
not in grade 11, students who were in grade 11 but 
age 16 or younger (born after September 30, 1967), 
and students who were in grade 11 but age 18 or older 
(born before October 1966). 



-67- 



85 



QED Data 



The Version 3.1 Public-Use Data Tapes contain several variables 
obtained from information supplied by Quality Education Data, Inc. 
(QED). These data, last revised by QED in March 1983, have been used 
for the following variables: 



Student Data Files 



ORSHPT 
NTEACHC 
SCHTYPE 
GRSPAN 
ENROLL 



IDP 
CAI 
URBAN 
NSTUDA 
NTEACHA 



SPECED 
LIB/MED 
INDARTS 
ADULTED 
PCENROL 



School Questionnaire Files 



SORSHPT 

SNTCHC 

SSCHTYP 

SGRSPaN 

SENROLL 



SIDP 

SCAI 

SURBAN 

SNSTUD 

SNTEA''H 



SSPECED 
SLIBMED 
SINDART 
SPCENRL 



Wi^h the exception of (S)ORSHPT, each variable is briefly defined 
in the data codebooks. The variables ORSHPT and SORSHPT are the 
Orshansky Percentile, an indicator of relative wealth which roecifies 
the percentage of district school-age children who fall below cb.e 
poverty line. 

NAEP Reports 

1983-84 assessment dat? were used in the production of the 
following NAEP reports (complete bibliographic references are included 
in Chapter VII): 

The Reading Report Card: Trends in Reading over F o' 'r National 
Assessments 1971-1984 (1985); 

The Writing Report Card: Writing Achievement \n American Schools, 
1984 (Applebee, Langer and Mullis, 1986a); and 

Writing: Trends Across the Decade, 1974-1984 (Applebee, Langer 
and Mullis, 1986b). 



For each of these reports. Table IV. 9 provides information 
regarding the type of data used (Spiral, Paced, Grade, Age), the use of 
plausible values, and whether or not all relevant assessment data is 
contained on the 1983-84 Public-Use D?La Tapes. 
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Table IV. 9 
NAEP Report Characteristics 



Abbreviated 
Report Title 


D^ta Types Used 


Plausible 
Values 
Used? 


Relevant Data 
on 1983-84 
User Tapes? 


Spiral/Paced 


Grade/Age 


"The Reading 
Report Card. . 


Spiral 


Age 


Yes 


No* 


Paced 


Age 


"NAEP: Writing 
Trends. . 


Paced 


Age 


No 


No* 


-'The Writing 
Report Card. . ." 


Spiral 


Grade 


Yes 


Yes 



Report includes data from previous assessments. 
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V. Issues to Consider in Conducting 
Statistical Analyses Using 1983-84 NAEP Data 




Issues to onsider in Conducting Statistical Analyses 
Using 1983-84 NAEP Dati 



A* Introductio n 

Potential users of NAEP data should be aware of the special 
properties of the NAEP database that affect the validity of 
conventional techniques of statistical inference There are two 
reasons that standard procedures for statistical inference should not 
be applied to the NAEP data without modification* First, a complex 
sampling scheme, rather than simple random sampling, was used to 
collect NAEP data. Second, because of the speciali2<»d methods used to 
estimate reading and writing proficiencies in NAEP, the resulting 
proficiency values have different properties from ordinary test scores. 

These issues are outlined below and discussed in detail in the 
followirg sections. Section B describes the procedures used in NAEP to 
account for the special properties of the NAEP data. Section C 
provides approximate procedures that are easier to implement and less 
expensive. Although the approximate procedures are less precise, it is 
expected that the resulting degree of accuracy will be acceptable to 
most users of the NAEP data. 

1. Properties c : NAEP Data that Result from Complex Sampling 



In the NAEP sampling scheme, students do not have an 
equal probability of being selected. Therefore, as in all 
complex surveys, each student has been assigned a sampling 
weight. The larger the probability of selection for students 
within a particular demographic group, the smaller the weights 
for those students will be. When computing descriptive 
statistics or conducting inferential procedures, one should 
veight the data for each student. Performance of statistical 
analyses without weights can lead lo misleading results. 

Another way in which the complex sample design used by 
NAEP differs from simple random sampling is that the NA'?P 
sampling scheme involves the selection of clusters of students 
who come from the same school, as well as clusters of schools 
that come from the same geographical region. As a result, 
observations are not independent of one another as they are in 
a simple random sample. Therefore, use of ordinary formulas 
for estimating the standard error of :>ample statistics, such 
as means, proportions or regression coefficients, will result 
in valu'*s that are too small. The standard error, which is a 
measure of the variability of a sample statistic, gives an 
indication of how well that statistic estimates the 
corresponding population value. It is used in conducting 
tests of statistical significance* If conventional simple 
random sampling (SRS) forrvlas are used to compute standard 
<5rrors, "too many" statistically significant results i/ill 
occur. 



-70- 




ERLC 



Yet another effect of the NAEP sampling scheme is a 
reduction of the effective degrees of freedom. In a simple 
random sample, the degrees of freedom of a variance estimate 
are based on the number of subjects. In the NAEP design, the 
degrees of freedom are a function of the number of Primary 
Sampling Units (PSUs) and the number of strata in the design, 
rather than the number of subjects (see Chapter III for a 
discussion of the sample design). Therefore, the ordinary 
formulas for obtaining degrees of freedom are not valid with 
the NAEP data. 

2. Properties of NAEP Data that Result from Proficiency 
Estimation Procedures 

In conventional applications of item response theory 
(IRT) scaling, the number of items administered to each 
respondent is sufficient ic obtain a precise estimate of each 
individual's proficiency. In NAEP, however, the goal is to 
estimate group means, rather than individual proficiency 
values. Some respondents may answer only a few questions. 
Procedures described in detail below are used to estimate a 
distrib' "ion of plausible values for each respondent and to 
draw va aes at random from this distribution. The resulting 
values are appropriate for calculating statistics based on 
certain groups, but do not represent precise estimates of 
proficiency for individual respondents. 

Use of this method of estimating proficiency results in 
an increase in the variability of statistics such as means and 
regression coefficients. Thus, there are two reasons that the 
standard errors of these statistics are larger than t-hs values 
that would be obtained with conventional formulas: the use of 
cluster sampling, which results in non-independent 
observations; and the use of proficiency estimation 
methodology that provides consistent estimates of selected 
group characteristics, but does not yield precise estimates 
for individual respondents. Another property of the 
proficiency estimates based on plausible values is that for 
some subgroups of respondents, mean proficiencies may 'a 
biased. This is explained in Section 2 below. 

Procedures Used by NAEP to Estimate Variability 

This section describes how the magnitude of variability from the 
sources described above was estimated in the NAEP reports. The Reading 
Report Card and The Writing Report Card . This estimate represents the 
most precise estimate obtainable, given the resources available to 
NAEP. Better estimators of uncertainty could be obtained with more 
resources. Because most secondary users will have fewer resources than 
those available for the NAEP reports. Section C provides less expensive 
approximations to the estimators described below. 
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Estimation of Uncertainty Due to Sampling 



A major source of uncertainty in the estimation of the 
value in the population of a variable of interest exists 
because information about the variable is obtained only on a 
sample from the population. To reflect this fact, it is 
important to attach to any statistic (e.g., a mean) an 
estimate of the sampling variability to be expected for that 
statistic. (The estimation of variability due to imperfect 
measurement, discussed in the following section, is also 
essential. ) 

Estimates of sampling variability are designed'' to provide 
information about how much the value of a given statistic 
would be likely to change if the statistic had been based on 
another, equivalent, sample of individuals drawn in exactly 
the same manner as the achieved sample. Consequently, the 
estimation oi the sampling variability of any statistic must 
take into account the design of the sample. 

The NAEP sample is obtained via a stratified multi-stage 
probability sampling design which includes provisions for 
sampling certain subpopulations at higher rates. Additional 
characteristics of the sample include adjustments for both 
nonresponse and post-stratification. The resulting sample has 
different statistical characteristics from those of a simple 
random sample. In particular, because of the effects of 
cluster selection (students within schools, schools within 
PSUs) and because of effects of nonresponse and 
post-stratification adjustments, observations made on 
different students cannot be assumed to be independent of each 
other (and are, in fact, generally positivaly correlated). 
Furthr^rniore, to account for the differential probabilities of 
selection (and the various adjustments), each student has an 
associated sampling weight, which musv He used in the 
computation of any statistic and which is itself subject to 
sampling variability. 

Treatment of the data as a simple random sample, with 
disregard for the special characteristics of the NAEP sample 
design, will produce underestimates of the true sampling 
variability. 

There are a number of techniques available to obtain a 
proper estimate of sampling Variability. The technique used 
for the NAEP reports, the jackknife method, is described in 
Chapter IV, Section H. The ETS jackknife technique assesses 
the sampling variables of any statistic as the sum of 
variability attributable to each of 32 pairs of PSUs. The 
variance attributable to a particular PSU pair measures how 
much the value of the statistic would change were the 
information in that PSU pair to change. A key property of 
this estimator of variance is that is accounts for both 
within-PSU sources of variability and belween-PSU variability. 
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It also accounts for stratification and the consequences of 
nonresponse adjustments and, for the multi-weight version, 
post-stratification adjustments. Chapter IV, Section H 
presents details concerning the single-weight and multi-weight 
jackknife procedures. A brief review of the (preferred) 
multi-weight procedure follows. 

The component of the sampling variability attributable to 
a PSU pair is estimated as the squared difference between the 
value of the statistic for the complete sample and a 
pseudo-replicate formed by recomputing the statistic on a 
specially-constructed pseudo-dataset. This pseudo-da taset is 
created from the original dataset by eliminating the first PSU 
of the pair and replacing it with a copy of the second PSU in 
the pair. For computational purposes^ the pseudo-dataset 
associated with a given PSU pair is the original dataset with 
a different set of weights, referred to as JKWTOl through 
JKWT32 on the data tapes, where JKWTi is for the i^ PSU pair. 
This set of weights allows measurement of the total effect of 
replacing the first PSU of the PSU pair with a copy of the 
second, including adjx^stments for nonresponse and 
post-stratification. The pseudo-replicate associated with 
the i^ PSU pair for a given statistic is obtained by 
recalculating the statistic using the weights JKWTi instead of 
the sampling weights. 

Specifically, let t(^,w) be any statistic which is 
a function of the sample responses ^ and the weights w and 
which estimates population value T. For example, t could be a 
weighted mean, a weighted percent-correct point — a weighted 
regression coefficient. The t(^,w), computed Wj.i.n the 
sampling weights (WEIGHT on the data tapes) is the appropriate 
sample estimate of T. To compute Var(t), the sampling 
variance for this statistic, proceed in the following manner: 

1. For each of the 32 PSU pairs compute the associated 
pseudo-replicate for the statistic. For the i 
pair, this is 

t. = t(y, JKWT i) 

which is the statistic t recalculated by using JKWTi 
instead of the sampling weights. 

2. The sample variance ot t is 



3 2 

Var(t) = Z (t^ - t)^ 

This estimation technique is called the multi-weight 
jackknife approach. Table VI. 10 provider SPSS-X and SAS code 



-73- 



for carrying out the above in the special case of a weighted 
mean. 

a. Degrees of Freedom of the Jackknifed Variance 
Estimate 

Note that the jackknife procedure estimate the 
sampling variability of the statistic by assessing 
the effect of change in the sample at the paired PSU 
level. For this reason, the number of degrees of 
freedom of the variance estimate Var(t) will be at 
most equal to the number of PSU pairs. The number of 
degrees of freedom, which indicates the variability 
of the variance estimate, equals the number of 
independent pieces of information used to generate 
the variance. In the current case, the pieces of ^ 
information are the 32 squared differences (t^ - t) , 
each supplying at ro^st one degree of freedom 
(regardless of how many individuals were sampled 
within any PSU). 

Increasing the number of individuals sampled 
within any PSU results in a lower estimat-^ of 
sampling variability because the within-PSU component 
is reduced. This, however, does not improve the 
estimation of the between-rsu component of 
variability, which depends on the number of PSUs 
selected • 

The number of degrees of freedom of the sample 
variance estimator can be less than the number of PSU 
pairs (32) when a few of the £;quared differences 
(t. - t)^ are markedly different in magnitude than 
the remainder. An extreme case of this is when one 
or more of the t. are identical to t, jo that 
(t. -t)=0. This may happen, for example, when 
the statistic t is a mean for a subgroup and no 
members of that subgroup come from the PSU pair 1. 
Such a PSU pair contributes zero degrees of freedom 
to the variance. 

An estimate of the effective nuir- er of degrees 
of freef'om for Var(t) comes from an approximation due 
to Satterthwaite (1946) (See Ccchran (1977), p. 96 
for a discussion) . 




The effective number of degrees of freedom using 
t^iis approximation is 



(t^ - t)')' 

A-l 



E (t, - t)' 

which is never larger than 32. 

Proficiency Values 

Jackknifing provides a reasonable estiue f uncertainty 
due to sampling from a finite population when the variable of 
interest is observed without error from every respondent. As 
noted in Section F of Chapter IV, however, some of the key 
reporting variables in NAEP are not observed without error. 
Although both reading proficiency and writing proficiency are 
construed to characterize individual respondents, these 
proficiencies are not observed directly from any respondent. 
They are instead inferred imperfectly from responses to a few 
reading or writing exercises. 

Each respondent provides answers to too few cognitive 
exercises to provid'" an accurate point estimate of his or her 
ability. However, as described in Chapter IV, Section F, it 
is possible to summarize what is known about the proficiency 
value 6 of respondent i given his or her responses to 
cognitive exercises (x^) and background variables (y.) in 
terms of a probability distribution p(9|x. ,y.). For 
computational convenience, these distributions have been 

approximated by a set of five "plausible values", 6^ through 

e , drawn at random from p(6|x^,y, ). They are labeled RDVALl 
through RDVAL5 and WRTVALl through WRTVAL5 on the data tapes. 
The spread of these plausible values reflects the uncertainty 
about the 9 value associated with that respondent given the 
observable variables x and y . The background variables y used 
in constructing these plausible values are age, grade, region 
of the country, parental education, sex, ethnicity, and size 
and type of community. 

Let t(9,y) be a statistic, or a function of the values of 
e and y in the sample, estimaf«ng a population value T. 
Examples of statistics t would be weighted means, percentile 
points, and regression coefficients. If 6 were observed 
directly for sampled pupils, it would be possible to 
approximate the precision of t through standard methods for 
survey samples, such as the jackknife technique described 
above; the result would be, say, Var(t). This value addresses 



( 
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uncertainly due to sampling only. Using plausible values, the 
additional uncertainty incurred when 8 is not observed 
directly can be managed in the following manner: 

1. Using the first vector of plausible values for each 
respondent (i»e*, RDVALl), evaluate t as if the 
plausible values were the true values of 6* Denote 

the result t^. 

2. Using the multiple weight jackknife approach, compute 

the estimated saunpling variance of t^, or Var(tj^), 
with respect to respondents' first vectors of 
plausible values* 

3» Carry out steps (1) and (2) for the second through 

fifth vectors of plausible values, thus obtaining t^ 

and Var(t^) for u = 2,...,5. 

4. The best estimate of t obtainable from the plausible 
values is the average of the five values obtained 
fro.n the different sets of plausible values: 

t- = E t /5 

u 

u 

5* An estimate of the variance of t. is the sum of two 
components: 



Var(t.) = Z Var(t^)/5 + E (t^ - t.)' /5 

u u 

The first component in Var(t.) reflects uncertainty 
due to sampling respondents from the population; the 
second component reflects uncertainty due to the fact 
that sampled respondents' 9s are not known precisely, 
but only indirectly through x, and y^* 

The f' t component in Var(t.) is attainable by jackknife 
methods for means as described in the preceding section* 
Table VI. 11 provides SPSS-X and SAS code to apply the steps 
given above in this special case. In principle, jackknifing 
could also be applied to more complicated statistics such as 
regression coefficients* 

Computations in this manner of statistics t, involving 
only writing or reading proficiency, in conjunction with the 
specific background variables y listed above, provides nearly 
unbiased estimaces of the population values T* Statistics 
involving proficiency and background variables not included in 
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this list are subject to biases, the magnitude of which depend 
in part on the relationship of the excluded backgrounr^ 
variables to che included background variables. (See the NAEP 
Technicai Report for details.) 

C. Approximations 

Estimating variability requires computing a statistic 165 times, 
including 33 runs to obtain an estimate and a variance estimate from 
each of the 5 sets of plausible values* Because the cost of the full 
procedure may be prohibitive in many studies^ approximate procedures to 
produce reasonable estimates at lover costs are provided below. 

1. Design Effects 

The major computational load in coinputiiig uncertainty 
measures for any statistic exists in the computation of the 
uncertainty due to sampling variability. As mentioned above, 
the procedure detailed in Section B.l requires that the 
statistic be recomputed 33 t5.mes to obtain an estimate of the 
sampling variance of the statistic. If the population value 
of interest is based on proficiency values, so that the 
statistic is computed on a set of plausible values, the entire 
process must be repeated 5 times, once for each set of 
plausible values. 

This section describes how to approximate the sampling 
variability for a statistic based on a single set of plausible 
values. Section C.2 provides way of reducing the number of 
computations in the case of estimation with sets or plausi'jle 
values. 

As indicated in Section B.l, it is inappropriate to 
estimate the sampling variability of any statistic based on 
the NAEP databa.se by ucing simple random sampling (SRS) 
formulas. These formulas, which are the ones used b> most 
standard statistical software such as SPSS and SAS, v^ll 
produce variance estimates which are generally much smaller 
than is warranted by the sample design. 

It may be possible to account approxijrately for the 
effects of the sample design by using an inflation factor, the 
design effect, developed by Kish vl967) and extended by Kish 
and Frankel (197A). The design effect for a statistic is the 
ratio of the actual variance of the statistic (*"al:ing the 
sample design into account) over the cciiventional variance 
estimate based on the St*me number of elements. To avoid 
sources of bias due to improper representation, this 
conventional estimate must use the sampling weights. The 
(*esign effect may be us:^d co adjust error estimates based on 
simple random sampling assumptions to account approximately 
€or the effect of the design. In practice, this is often 
accomplished by dividing the total sample size by the design 
effect and using this effective sample size in the computation 
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of errors* Note that the value of the design effect depends 
on the type of statistic computed and the variables considered 
in a particular analysis as well as the clustering effects 
occurring among sampled elements* 

Based on empirical results and theoretic considerations, 
Kish and Frankel (1974) have developed several conjectures 
about design effects: 

1. Generally, the design effects for complex statistics 
from complex saunples are greater than 1, causing 
variances based on simple random sampling assumptions 
to tend to be underestimates* 

2. The design effects for complex statistics (such as 
regression coefficients) tend to be smaller than the 
corresponding design effects for means of the same 
variables. Hence, these latter ^scimates, which are 
more easily computed, tend to give overestimates of 
the design effects of complex statistics. 

3. The design effects of complex statistics tend to 
resemble those of means; variables with a high design 
effect of the mean also tend to have high design 
effects for complex statistics involving those 
variables. 

To incorporate the design effect idea in a statistical 
analysis, proceed in the following manner: 

1. For a given class of statistics (e.g., means, 
percentile points, regression coefficients), compute 
the jackknife variance as in Section B.l for a number 
of cases. The cases should cover the range of 
situations for which the approximation is to be used. 
If various subpopulations are to be considered, it is 
important to have information on the relative 
variability within each subgroup. This is especially 
important if certain subgroups are more highly 
clustered in the sample. 

2. For the identical cases, compute the conventional 
estimate of the variance. This estimate must take 
the sample weights into account to avoid problems of 
bias due to improper representation. To account 
properly for the difference between the number of 
individuals being sampled and the total of the 
sampling weights, the weights should be scaled so 
that their sum equals the sample size. 
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3* For each case^ compute the design effect where the 
design effect for case j is 



the ratio of the jackknife variance estimate of the 
statistic to its conventional variance estimate. 

4. If the design effects for the various cases are 
tolerably similar^ choose an overall composite design 
effect. If the design effects for certain subgroups 
appear to cluster around a markedly different value 
from the remaining cases ^ treat those subgroups 
separately. 

5. In the cas" that a consistent overall design effect 
has been found: 

(1) rescale the weight of each individual so 
that the sum of the scaled weights is equal 
to the effective sample size 



(2) conduct a traditional weighted analysis 
using these scaled weights 

6. The degrees of freedom for any variance estimates 
obtained by using this approach is still at best 32, 
the number of PSU pairs, as it was for the jackknife. 
Accordingly, tests of significance produced by 
standard programs (which will use the effective 
sample size minus the number of parameters for error 
degrees of freedom) should be interpreted with 
extreme caution because they are likely to be too 
liberal. Significance and inferential procedures are 
properly based on the smaller error degrees of 
freedom (32). 

Multiple Runs with Different Imputes 

A second method of reducing costs is to use fewer runs on 
plausible value sets. A statistic computed from a single set 
of plausible values has the same expectation as the averages of 
the five, but does not take into account the uncertainty 
surrounding 9 values. Use of at least two, but less than 
five, sets of plausible values to evaluate a statistic will 
properly account for this uncertainty and will reduce costs at 
the same time. The occurrences of "5" in the procedure 
outlined above would be replaced by a "2", "3", or "4" as 



N 



^ _ sample size 
•ff ~ design effect 
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appropriate* The resulting decrease in computation is 
accompanied by a decrease in precision for estimating Var(t ). 

Note: It is not appropriate to average the five 
plausible values associated vith each respondent and~analy2e 
thos3 averages ♦ The result of this computaTion is not 
generally equal to the correct value ^ 

3* Implementing the Full Procedure 

Full implementation of the procedure employed by NAEP 
requires 165 computations of a statistic* Employing the 
suggestions described above reduces computation considerably. 
If design effects are used to account for variation due to 
sampling (Step 2 of the procedure outlined above), no more 
than 5 calculations need be performed* In any event, at least 
two calcu. ations should be performed to account for 
uncertainty due to 0 and uncertainty due to sampling variance. 

A Note Concerning Multiple Comparisons 

In performing multiple hypothesis tests it is important to 
consider the fact that if J tests are performed, each with a Type I 
error rate (the probability of rejecting the null hypothesis when the 
null hypothesis is true) of a, the Type I error rate for the entire set 
of contrasts could be as high as Ja. Therefore, it is desirable to use 
a multiple comparison procedure to control the overall error rate for 
the entire set of hypothesis tests. In the present case, it is 
advantageous to use a procedure that allows control of the error rate 
for sets of varying size that may include both palrwise and complex 
comparisons. (An example of a complex contrast is a comparison of one 
group to the average of two other groups.) The Dunn-Bonferroni 
approach is, therefore, a good choice. To apply this method in its 
simplest form, we need only decide at what level we wish to control the 
setwise error rate (a^) and than set the Type I error rate for each 
comparison equal to = a^/J, where J is the number of comparisons. 

For example, suppose we wanted to perform three pairwise 
comparisons between regional groups , as well as one complex comparison, 
controlling at .05. The Type I error rate for each comparison 
should be set at = a^/J = .05/4 = .0125. The required critical 
value can be obtained from a table of the Bonferroni t-statistic 
(Miller, 1981, p. 238) with the appropriate degrees of freedom. 
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VI* The 1983-84 Public-Use Data Tapes : 
Contents and Format 



Introduction 

This chapter provides detailed information concerning the 
contents and format of each data file on the Public-Use Data 
Tapes and of the printed layouts and codebooks produced for the 
1983-84 assessment* In reviewing the material below, readers may 
want to refer back to the detailed information concerning reading 
proficiency, writing proficiency and standard errors contained in 
Chapter IV. 

Raw Data Files 

The raw data files contain one record per student, excluded 
student, teacher, and school. All raw data files are rectangular, 
so that record lengths are fixed and a given variable always 
occurs in the same position on every record within a file. The 
data files for each grade/age are described in Tables VI. 13, 14, 
and 15. 

As explained in Chapter III, and due to the BIB Spiralling 
design, students were administered different blocks of exercises 
during the 1983-84 assessment. As a result, each student record 
contains blank spaces for the exercise blocks which were not 
included in his or her assessment booklet (missing by design). In 
addition, fields are blank for items which did not appear in booklets 
due to a printing error and for the professionally-scored items which 
were not included in reliability checks (see Chapter IV). 

The following codes have been included in the raw data files, 
codebooks, and machine-readable catalogs: 

Table VI. 1 Code Definitions 



Code for 

Field 
Width = 1 


Code for 

Field 
Width = 2 


Code Definition 


5 


55 


Illegible 
(professionally-scored items only) 


6 


66 


Off Task 

(professionally-scored items only) 


7 


77 


I Don't Know 


8 


88 


Omitted or Not Reached 


9 


99 


Out-of-Range Response 
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C* Layouts 

With the exception of field types, all information in the 
layouts has been numerically coded ♦ The layouts include the 
following information for each data field: 

Seq> No^ (Sequence Number) : Fields are numbered sequentially to 
represent the order in which they appear on the data record. 

Field Name : A short name (of eight characters or less) which 
identities the field* The format of the field names associated 
with response data is explained below: 

Position 1 identifies nature/source of the response data: 

B = Common background item within common background 
block 

S = Background and attitude item within cognitive 
block 

N = Cognitive item within cognitive block 

C = School questionnaire item 

T = Teacher questionnaire item 

X = Excluded student questionnaire item 

Positions 2 through 5 identify an exercise (Student 
files) or question (School, Teacher, Excluded Student 
files) 

Positions 6 through 7 identify a part within an exercise 
(Student files) or a part within a question (School, 
Teacher, Excluded Student files) 

Position 8 identifies the block containing an item 
(Student BIB Spiral files) or tape containing an item 
(Student Paced Tape files) 

Col. Pos. (Column Position) : The start location of the field on 
the data record. 

Field Width : The number of characters in the field. 

Decimal Places : The number of digits to the right of the decimal 
point in the field. The raw data contains implicit and explicit 
decimal points. 

Type : The files include three field types, as follows: 

Type A (Alpha) fields designate free-response, 
alpha-numeric data. For example, several 
multiple-choice items included both fixed responses 
which a student could circle and a portion for "other" 
responses which a student could fill-in. The fill-in 
portion of the item is a Type A field. (The fixed, 
multiple-choice portion of the item is a Type D field). 
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Type C (Continuous) fields designate continuous 
numerical data without fixed ranges* 

Type D (Discrete) fields designate discrete data with a 
fixed number of responses. Type D fields may include 
raw item responses or imputed (derived) categorical 
variables* 

Type PI (Discrete with "I Don't Know") fields designate 
discrete data with a special code for an "I Don't Know" 
response. Depending on the field width, the "I Don't 
Know" value is either 7 or 77. 

Range : The range of values or of valid responses for a field. 

Key Value ; The correct response for a multiple-choice item. 

Short Label : A brief description of the information in the field. 



Codebooks 

Bach entry in the codebooks represents a distinct assessment 
item or item information (i.e., weights). The first line of each 
codebook entry contains the following information: 

Seq. No. ( Se quence Number) : In conjunction with the numbers 
assigned in the layouts, the fields in the codebooks are numbered 
sequentially. 

Field Name : A brief description of the information in the field. 
The field names in the codebooks correspond to those in the 
layouts. 

Rel. Ind. (Released Indicator) : Indicates that an item has been 
released to the public ("R") or that an item has been held secure 
("U"). 

Type; In conjunction with the four field types defined for the 



layouts above, the field type is designated afr' Alpha ("A"), 
Continuous ("C"), Discrete ("D"), or Discrete with "I Don't Know" 
("DI"). 

Block : Indicates that information appears in either an ID block 
or a weight block (WT) or, for assessment items, indicates the 
block in which an item appeared for the grade/age group of 
students for which the codebook was prepared . (See Table VI. 2 for 
block definitions). 

Item No. : Indicates the number of an item for the grade/age 
group of students for which the codebook was prepared . 
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Ages : Indicates the grade/age student groups to whom an item was 
administered, as follows: 

1 = Grade 4/Age 9 students; 

2 = Grade 8/Age 13 students; and, 

3 « Grade 11/Age 17 students. 

Please note that, for items administered to more 
than one grade/age group , the information under 
block and item number refers only to the 
grade/age group for which a codebook was pre- 
pared. For example, the block location and item 
number for an item in the grade A/age 9 
(grade/age 1) codebook apply to grade/age 1 only. 
The block location and item number may be differ- 
ent for grade/age groups 2 and 3. The block and 
item numbers for grade/age groups 2 and 3 are 
indicated in the appropriate codebooks. 

Name/Description : Provides a brief description of the item or 
information in the field. 

Old ID (NAEP ID Number ): If an item has been used in the past, the 
ID number assigned to it during previous assessments is included 
as the last number on the first line of the codebook entry. 

For all discrete variables and items, the third and subsequent 
lines contain each valid data value, its associated label, and the 
unweighted frequency of that value in the data file associated with the 
codebook. (For cognitive items, the key data value is indicated by an 
asterisk.) The last line under each discrete variable entry contains 
the "TOTAL" or sum of the frequency counts as an extra check for 
analyses . 

For items within a given grade/age cohort for which IRT estimates 
have been calibrated, the third through fifth lines also contain the 
IRT parameters and their values: parameter "a" (discrimination); 
parameter "b" (difficulty); and parameter "c" (lower asymptote). 

Machine-Readable Catalogs 

Each raw data file has a corresponding machine-readable file; 
these files all have the suffix CAT. The files, designed to help users 
to extract data of interest from the raw data, contain a record for 
each variable or item on the raw data files. Table VI. 3 contains the 
Kachine-readable catalog data layout. Specific information concerning 
the contents of the catalogs is provided below. 

Field Sequence Number : Fields are numbered sequentially to represent 
the order in which they appear on the raw data record. 

Field Name: A short name of eight characters or less that identifies 
the field. 
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Table yi. 2 
Block Definitions 







np<;rr 1 o t "i on 


ID 


Student Files 


Assessment identification and 

1 iixuL nia L iviii xi.uni uvjuivxcl v.\ivd. 


WT 


Student Files 


Weights and derived variables 


CB 


Student Files 


Common background items 


BA - BY 


Student Files 


RTR AnH ITBTR i tarn hlnrlrc 

(note: there is no block BI) 


CP 


\3 1. CI vA c / ^ 

Paced Files 


wUlUUiUil wO^IXAXvUiiU X LwlilO ^L/^^XXX^ 

to Grade 11/Age 17 Paced Tapes 


PB 


Paced Files 


Paoorl Tadp hApWarT~DtinH and 

attitude items 


Pi - P4 


r a^cvi r x xco 


Paced cognitive items; corres- 

•^nnH ffi ParPfl Taop<; thrmiffh 
\* f f jLcopc^^xv cxjr 


XT 

Ax 


£iA^X UUCU 

r 1 xco 


AQQPQQmAnf 4 rfpn f 4 ^ "1 PA f 4 fin Anfl 

no OCO Ollldl L XUdlLXXX^ClLXvll ClilU 

4 n f fii*mA f 4 fin frfim oiiP** tionnAi t*p 

X ilXVi^ Ilia LXViil XLvlil V| Vl^O k X vliiiGl X L ^ 


xw 


Excluded 
student Files 


Weights and derived 
variables 


XQ 


Excluded 
Student Files 


Excluded student questionnaire 
i tems' 


TI 


Teacher 
Files 


Assessment identification and 
information from questionnaire 
cover 


TW 


Teacher Files 


Weights and derived variables 


TQ 


Teacher Files 


Teacher questionnaire items 


SI 


School Files 


Assessment identification 


SW 


School Files 


Weights and derived variables 


SQ 


School Files 


School questionnaire items 
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Table VI. 3 



NAEP 1983-84 Machine-Readable Catalog Data Layout 



Start End 
Coli'.mns 


1 Field 
Width 


Field Description 


Comments 


1-4 


4 


Field Sequence Number 




5-12 


8 


Field Name 


New NAEP Identification 


13 - 16 


4 


Start Column 




17 - 20 


4 


End Column 




21 - 22 


2 


Field Width 




23 - 23 


1 


Decimal Places 




24 - 24 


1 


Field Type 


l=Dlscrete 

2aContinuous 

3=Alpha 


25 - 26 


2 


Number of Valid Responses 


Excluding Multiple, 

Out-Of-Range and 

I Don't Know Responses 


27 - 28 


2 


Correct Response 




29 - 30 


2 


I Don't Know (IDK) Code 




31 - 32 


2 


No Response Code 




3S - 34 


2 


Multiple and Out-0£-Range 
Code 




35 - 84 


50 


Field Label 




85 - 100 


16 


Old NAEP Identification 




101 - 124 


24 


IRT Parameters 


Three fields, 

ei^ht positions each 


125 - 126 


2 


Number of Response 
Values and Labels 


This number determines the 
number of response values 
and labels that follow 


127 - 128 


2 


Response Value 


1st Value 


129 - 148 


20 


Response Label 


149 - 150 


2 


Response Value 


2nd Value 


151 - 170 


20 


Response Label 


• 
• 
• 




• 
• 
• 


• 
• 


1645 - 1646 


2 


Response Value 


70th Value 


1647 - 1666 


20 


Response Label 
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Start Column : The start location of the field on the raw data 
record ♦ 



End Column : The end location of the field on the raw data rooord* 

Field Width *, The number of characters in the field • 

Decimal Places : The number of digits to the right of the decimal 
point in the field* The raw data contains implicit and explicit 
decimal points* 

Field Type : The files include three field types, as follows: 

Type 1 (Discrete) fields designate discrete data with a 
fixed number of responses* Type 1 fields may include raw 
item responses or imputed categorical variables* 

Type 2 (Continuous/ fields designate continuous 
numerical data without fixed ranges* 

T ypa 3 ^Alpha) fields designate free-response, 
alpha-numeric data* For example, several multiple-choice 
items include both fixed responses which a student could 
circle and a portion for "other" responses which the 
student could fill in* The fill-in portion of the item 
is a Type 3 field* (The fixed, multiple-choice portion 
of the item is a Type 1 field*) 

Number of Valid Responses : The number ol valid responses for an item, 
excluding multiple, out-of-range, and I Don't Know responses* 

Correct Response : This is a two position field* If the second 
position is blank, the item has a unique correct response in the first 
position* If the second position is not blank, the item has more an 
one correct response and the field indicates the correct response 
range* The first position is the lowest correct response value; the 
second is the highest correct response value* For example, if possible 
responses for a j^iTofessionally-scored writing item ranged from 0 to 5, 
and 3 to 5 was considered an acceptable response, ^'he first position of 
the field would contain a 3 and the second position would contain a 5* 

I Don' t Know : For appropriate multiple-choice it^ms, the numbers in 
the "IDK Value" column reflect the option to respond by indicating "I 
Don't Know*" "I Don't Know" values have been recoded to 7 or 77* 
Otherwise, this field is blank* 

No Response Code : Where applicable, omitted o. not reached values have 
been recoded to 8 or 88* Otherwise, this field is blank* 

Multiple and Out-of-Range Cod e: Wnere applicable, multiple or 
out-of-range responses have been rcacoded to 9 or 99. Otherwise, this 
field is blank* 

Field Label : A 50-character description of the jtem or variable. 
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Old NAEP Identitication : If an item has been used in the past, this 
is the number assigned to it during previous assessments. 



IRT Parameters : Three eight-character fields containing IRT item 
parameters: parameter "a" (discrimination); parameter "b" 
(difficulty); and parameter "c" (lower asymptote). Each parameter is 
represented to a precision of five decimal places with an explicit 
decimal point. 

Number of Response Values and Labels : The number of possible valid 
responses plus "I Don't Know," no response, multiple and out-of-range. 

Response Values and Labels : For each possible item response, a 
two--position field which indicates the response value and a 20-position 
text field which provides a brief description of the response. 



SPSS-X and SAS Control Statement Files 

All data files on Version 3.1 Public-Use Data Tapes are 
accompanied by separate control files to facilitate the creation of 
SPSS-X and SAS system files. These control files include statements 
for variable definitions, variable labels, missing value codes, value 
labels, and an optional section for creating and storing scored 
variables. Each set of control statements also generates unweighted 
descriptive statistics of the reporting variables for the related data 
file and a listing of the contents of the saved system file. 

Specific details on the structure and use of these control 
files will be included in appropriate system sections. The common 
features of both types of control files, as well as general guidelines, 
are provided below. 

1. Variable Definition : The field names are listed in the order 
in which they appear on the file, along with their column 
position and input formats. If the field is numeric with no 
decimal places, no format is provided. Otherwise the format 
is indicated by a number for the number of decimal places, or 
a or '(A)' for a non-numeric field. 

2. Variable Labels : A 40-character text description for each 
field. 

3. Missing Values :' All blank fields in the data are 
automatically set tc the systeip missing value by each package. 
However, all multiple-choice and some open-ended items were 
prone to either multiple or out-of-range responses. These 
items were coded as fields of 9s in the data files. The 
control statement files instruct each system to treat these 
values as missing. 
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4. Value Labels : All numeric fields with discrete (or 
categorical) values are provided with 20-character text 
descriptors for each value within the variable's range. 

5. Scoring : For each item with one or more correct responses, 
control statements are provided for creating a scored 
variable, its label, and its value labels. The scoring of 
each item is performed according to the following scheme: 
missing values are copied as is, correct response values are 
recoded to 1, all other values, including No Response and "I 
Don't Know," are recoded as 0. The scoring of the No Response 
and "I Don't Know" values are coded separately from other 
incorrect responses to allow the user to edit these control 
statements and substitute alternate values. 

The scored variable naunes are derived by replacing the value 
in the sixth and seventh digits of the original name with that 
value plus 50. For example, the scored version of item number 
N001503A is N001553A. The entire block of scoring control 
statements is performed conditionally and, by default, will 
not be saved on the system file. The user must edit only one 
statement in either type of file to invoke the scoring and to 
save the new variables. 

IMPORTANT NOTE: The system file generation programs cannot run if 
both the control statement file and its corresponding data file reside 
on the same tape. Both SPSS-X and SAS will try to read a data file 
be*:ore they have completed processing the control statement file, which 
is physically impossible if both files are on the same tape. It is 
advisable that the user copy the control files to disk, as they require 
less storage space and allow the user to edit the control statements 
before generating the system files. 
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Creating SPSS-X System Files 



Each SPSS-X control statement file is linked to its 
corresponding data file through the file name: the suffix DAT in the 
data file name is replaced by SPX to obtain the control statement file 
name. For example, file "Y15RDWR1 .BIB. SPX" is the control statement 
file for data file "Y15RDWR1.BIB.DAT." 

All SPSS-X control statement files have been generated according 
to the following structure: 

Table VI. 4 SPSS-X Control Statement Synopsis 



TITLE 

label for sysout of file generation run 
FILE LABEL 

label to be stored with file 
DOCUMENT 

text description of data to be saved in file 
DATA LIST FILE=:RAWDATA/ 

variable names, locations, & formats 
VARIABLE LABELS 

40-character label for each variable 
MISSING VALUES 

list of variables to have user-missing values assigned 
VALUE LABELS 

variable names, values, & value labels 
TEMPORARY [delete this statement to 

save scored variables] 

RECODE 

oldvar (SYSMIS=SYSMIS) (9=9) (keyval=l) 
(nrval=0) (idkval=0) (ELSE=0) INTO newvar 



VARIABLE LABELS 

labels for new variables 
MISSING VALUES 

for multiple & out-of-range recodes 
VALUE LABELS 

l=Correct O=lncorrect 
FREQUENCIES 

reporting variables 
SAVE OUTFILE=SYSFILE/COMPRESSED 
DISPLAY LABELS 
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The TEMPORARY command instructs SPSS-X to perform the subsequent 
scoring statements on a temporary basis and delete the new variables 
after the next procedure encountered (FREQUENCIES). Thus, the scored 
variables will NOT be saved on the system file unless the TEMPORARY 
command is commented or edited out. 

All control statement files assume that the file handle (or 
DDNAME) for the input data file is RAWDATA, and the file handle for the 
output system file is SYSFILE. 

The control statements were coded according to the command and 
procedure descriptions in the SPSS-X User^s Guide (SPSS Inc., 
McGraw-Hill Book Company, 1983)"! They were tested under SPSS-X Version 
2.1 (IBM-OS/MVS). 



Creating SAS System Files 

Each SAS control statement file is linked to its corresponding 
data file through the file name: the suffix DAT in the data file name 
is replaced by SAS to obtain the control statement file name. For 
example, file "Y15RDWR1.BIB.SAS" is the control statement file for data 
file "Y15RDWR1.BIB.DAT." 

All SAS control statement files have been generated according to 
the following structure: 
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Table VI. 5 SAS Control Statement Synopsis 



TITLE 

DATA SYSFILE.xxx; 
INFILE RAWDATA; 
INPUT 

Variable names, colvmn positions, & formats 
LABEL 

40-character variable labels 
ARRAY MVn (I) 

list of variables to be recoded for missing 
*D0 OVER MVn; 

* IF MVn=7 THEN MVn=.; 

* IF MVn=8 THEN MVn=.5 

* IF ^Wn=9 THEN MVn=.5 

* END; 
%MACRO FJ1TVAR5 

PROC FOilMAT; 
VALUE 

variable name value=value label 



• 

ZMEND FMTVAR; 
LENGTH DEFAULT=2 

other variables with appropriate lengths; 
ZMACRO SCORE; 
%MACRO RECODE; 

SAS macro to perform scoring for each variable 
%MEND RECODE; 
%RECODE 

(oldvar , newvar , idkval , nrval , keyval) 



• 

LABEL 

labels for new variables 
PROC FORMAT; 

value labels for new variables 
ZMEND SCORE; 

*«SCORE; [delete asterisk to 

save scored variables] 

%FMTVAR; 
PROC FREQ; 
TABLES 

reporting variables 
PROC CONTENTS NCSOURCE POSITION; 
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All SAS control files use the SAS Macro Language facility to 
reduce the number of source statements generated and to provide 
consistent performance of repetitive functions. Therefore, i:he user 
must ensure that the MACRO option is invoked whenever processing any of 
the control statement files. 

The DO OVER through END statements following each ARRAY statement 
set up the conversion of the "I Don't Know," No Response, and Multiple 
Response codes to the system missing value. However, once this 
conversion is executed and saved on the system file, these recoded 
values will be indistinguishable from actual missing values on the 
original data file. For this reason, these statements have been 
commented out to allow the user to decide which, if any, of the values 
are to be recoded. To activate the recoding, delete the asterisks 
preceding the DO OVER and END statements, and from the appropriate IF 
THEN statement(s). 

The SCORE MACRO sequence sets up the creation of scored variables 
and their labels. To save these variables on the system file, edit the 
statement immediately following the MACRO and delete the asterisk in 
the first column. 

The control statements were coded according to the command and 

procedure descriptions in the SAS User's Guide: Basics 1982 Edition 

(SAS Institute Inc., 1982). They were tested under SAS Version 82.4 
(IBM-OS/MVS). 



Merging Files Under SPSS-X or SAS 

The NAEP data files are structured to facilitate matching among 
the four instruments (Student, Teacher, School, and Excluded Student). 
However, for the purposes of analysis and reporting, only four types of 
linkages are valid: 

1) School with Student (BIB Spiral and Paced Tape) 

2) Teacher with Student (BIB Spiral only) 

3) School with Excluded Student 

4) School with Teacher with Student (BIB Spiral only) 

The primary linkage on all files is through the scrambled PSU and 
school code fields: SCRPSU, SSCRPSU, TSCRPSU and XSCRPSU. The secondary 
linkage for the BIB Spiral and Teacher files is through the scrambled 
teacher code fields: SCRTC and TSCRTC. All files are sorted by these 
fields to permit direct match-merging without the need to re-sort. 

When a hierarchical file match is performed, both SPSS-X and SAS 
build a rectangular file at the level of the lowest file in the match. 
Each record from the higher order file is repeated across the 
corresponding records of the lower order file. For example, in matching 
School with Student data, the information from one school record is 
repeated across all student records belonging to that school. Clearly, 
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the number of variables from the higher order file will have a greater 
impact on the size of the resulting merged file. 

The examples contained in Tables VI. 6 through VI. 9 will perform 
direct matches according to the four linkages listed above. The KEEP 
statements are not necessary to the performance of the merge, but when 
applied to only those variables required for analysis will make more 
efficient use of computer resources. These examples also assume thac no 
transformations are to be performed on the input files. If 
transformations are desired for analysis, the most efficient course to 
follow would be to transform the variables from the higher order file 
first, perform the match procedure, then transform the variables from 
the lower order file. 



Table VI. 6 
Matching School and Student Files 



SPSS-X 

MATCH FILES 
TABLE=SCHOOL/ 

RENAME=(SSCRPSU=SCRPSU)/ 
FILE=STUDENT/ 

KEEP=SCRPSU, other school & student variables/ 
BY=SCRPSU 



SAS 

DATA HATCHl; 

MERGE SCHOOL (RENAME=(SSCRPSU=SCRPSU) 

KEEP=SSCRPSU other school variables) 
STUDENT(KEEP=SCRPSU other student variables); 
BY SCRPSU; 
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Table VI. 7 
Matching Teacher and Student Files 



SPSS-X 

MATCH FILES 

TABLE=TEACHER/IN=INTEACH/ 

RENAME=(TSCRPSU=SCRPSU)(TSCRTC=SCRTC)/ 
FILE=STUDENT/ 

KEEP»SCRPSU,SCRTC, other teacher & student variables/ 
BY»SCRPSU,SCRTC 
SELECT IF (INTEACH) 



SAS 

DATA MATCH2; 

MERGE TEACHER(RENAME=(TSCRPSU=SCRPSU TSCRTC=SCRTC) IN=T 

KEEP=TSCRPSU TSCRTC other teacher variables) 
STUDENT(KEEP=SCRPSU SCRTC other student variables); 
BY SCRPSU SCRTC; 
IF T; 
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Table VI. 8 

Matching School and Excluded Student Files 



SPSS-X 

MATCH FILES 
TABLExSCHOOL/ 

RENAME=(SSCRPSU=SCRPSU)/ 
FILE=>EXCLUDE/ 

RENAME. (XSCRPSU=SCRPSU)/ 
KEEP=SCRPSU, other school & excluded student variables/ 
BY=SCRPSU 



SAS 

DATA MATCH3; 

MERGE SCHOOL (RENAME=>(SSCRPSU=SCRPSU) 

KEEP=SSCRPSU other school variables) 
EXCLUDE (RENAME=(XSCRPSU=SCRPSU) 

KEEP=SCRPSU other excluded student variables); 
BY SCRPSU; 
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Table VI. 9 

H atching School, Teacher and Student Files 



SPSS-X 

MATCH FILES 

TABLE«SCHOOL/IN=INSCHOOL/ 
RENAME- ( S SCRPSU.SCRPSU ) / 

FILE-TEACHER/ 

RENAME-dSCRPSU.SCRPSU ) (TSCRTC=SCRTC ) / 

KEEP«SCRPSU,SCRTC, other school & teacher variables/ 

BY«SCRPSU 
SELECT IF (INSCEOOL) 
MATCH FILES 

TABLE-*/IN-INTEACH/ 

FILE=STIIUENT/ 

Ki-'EP=SCRPSU,SCRTC, other school, teacher & student variables/ 
BY^ ^CRPSU.SCRTC 
SELECT IF (INTEACH) 



DATA MATCH4A; 

MERGE SCHOOL (RENAME=(SSCRPSU=SCRP5U) IN=S 

KEEP=SSCRPSU other school variables) 
TEACHER(RENAME=(TSCRPSU=SCRPSU TSCRTC=SCRTC) 

KEEP=TSCRPSU TSCRTC other teacher variables); 

BY SCRPSU; 
IF S; 
DATA MATCH4B; 

MERGE MATCH4A (IN-T 

KEEP=SCRFSU SCRTC other school 

& teacher variables) 
STUDENT(KEEF-^SCRPSU SCRTC other student variables); 
BY SCRPSU SCRTC; 
IP T; 



-98- 

118 



J* Computing the Estimated Variance of a Mean (Jackknif ing) 
Using SPSS-X or SAS • 



Three methods for computing the estimated variance of a mean are 
presented in this section in SPSS-X and SAS program code form (see 
Chapter IV, Section H for discussion of the jackknife procedure). The 
first method given is the multi-weight method which may be UJied for any 
variable except the plausible reading and writing values. The second 
method is the multi-weight method to be used for the plausible values, 
making a correction for the variance in estimating the values. The 
third method is a single-weight method which is less computationally 
intense, but not as exact as the multi-weight method. 

The technique used for all three uiethods is essentially the same: 
for each variable to be jackknifed, generate two vectors of weighted 
sums and products. Sum these vectors across the entire file using the 
AGGREGATE (SPSS-X) or SUMMARY (SAS) procedures;. From the weighted sums 
compute the weighted means and thence compute the estimated variance 
and standard error. 

One advantage to this aproach is that it will accomplish the 
computation in one pass of the data. Another advantage, afforded by the 
AGGREGATE (SPSS-X) and SUMMARY (SAS) procedures, is the facility to 
compute subgroup statistics by using the BREAK keyword with the 
variable(s) defining the subgroups. All computations performed 
subsequent to the aggregation procedure are performed on each record of 
the collapsed file, corresponding to one of the subgroups. In the 
examples contained in Tables VI. 10, 11, and 12, the variable SEX is 
used as a break control variable, and the derived statistics are 
printed for each sex code. 

In Tables VI. 10 and VI. 12, the variable X may be any variable or 
transformation of variables except the plausible reading and writing 
values. In Table VI. 11, the vector or array named VALUE refers to the 
set of plausible reading or writing values. 
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Table VI. 10 

Standard Error Computation: Multi-Veight Method 



SPSS-X 

GET FILE-STUDENT/ 

KEEP-SEX, WEIGer.JKWTOl TO JKWT32,X 
VECTOR WT-JKWTOl TO JKWT32 
VECTOR WX(32) 
SELECT IF (NOT SYSMIS(X)) 
COMPUTE UTX-WEIGHT*X 
LOOP #1-1 TO 32 
. COMPUTE WX(#I) - WT(#I)*X 
END LOOP 

AGGREGATE OUTFILE=*/BREAK=SEX/UVN=N(VEIGHT)/ 

SWT,SW1 TO SW32 - SUM(WEIGHT, JKWTOl TO JKWT32)/ 
SWX.SXl TO SX32 - SUM(WTX,WX1 TO WX32)/ 

VECTOR SW - SWl TO SW32 

VECTOR SX - SXl TO SX32 

COMPUTE XBAR = SWX/SWT 

COMPUTE XVAR = 0 

LOOP #1-1 TO 32 

. COMPUTE #DIFF = SX(#I)/SW(#I) - XBAR 
. COMPUTE XVAR = XVAR + #DIFF * #DIFF 
END LOOP 

COMPUTE XSE = SQRT(XVAR) 

PRINT FORMATS XVAR, XSE (F8.4) 

LI ST VARIABLES-SEX , UWN , SWT , XBAR , XVAR , XSE 

FINISH 
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Table VI. 10 



(continued) 



SAS 

DATA A; 

SET STUDENT; 

ARRAY WT JKWT01-JKWT32; 

ARRAY WX WX1-WX32; 

IF (X NE .); 

WTX - WEIGHT*X; 

DO OVER WT; 
WX . WT*X; 

END; 
PROC SUMMARY; 

CLASS SEX; 

VAR WEIGHT JKWT01-JKWT32 WTX WX1-WX32; 
OUTPUT OUT.B N(WEIGHT)=:UWN 

SUM(WEIGHT WTX JKWT01-JKWT32 WX1-WX32)= 
SWT SW1-SW32 SWX SX1-SX32; 

DATA C; 
SET B; 

ARRAY SW SW1-SW32; 
ARRAY SX SX1-SX32; 
XBAR = SWX/ SWT; 
XVAR =x 0; 
DO OVER Sw; 

DIFF = (SX/SW)-XBAR; 

XVAR = XVAR+DIFF*DIFF; 
END; 

XSE = SORT (XVAR); 
PROC PRINT; 

VAR SEX UWN SWT XBAR XVAR XSE; 
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Table VI. 11 

Standard Error Computation! Multi-Weight Method 
(vith Co>;rection for Imputation) 



SPSS-X 

GET FILE.STUDENT/ 

KEEP.SEX, WEIGHT, JKUTOl TO JKWT32,REDVAL1 TO REDVAL5 
VECTOR VALUE.REDVALl TO REDVAL5 
VECTOR VT-JKWTOl TO JK!7T32 
VECTOR WX(32) 
VECTOR WS(5) 

SELE'TT IF (NOT SYSMIS(REDVALl) ) 
COMPUTE WTX-WEIGHT*REDVAL1 
LOOP #1.1 TO 32 

. COMPUTE WX(#I) =. WT(#I)*REDVAL1 

END LOOP 

LOOP #I»1 TO 5 

. COMPUTE WS(#I) = VALUE (#I)*WEIGHT 
END LOOP 

AGGREGATE OUTFILE=*/BREAK=SEX/UUN=N(WEIGHT)/ 

SWT.SWl TO SW32 » SUM ( WEIGHT, JKWTOl TO JKWT32)/ 
SWX.SXl TO SX32 » SUM(WTX,WXl TO WX32)/ 
SSI TO SS5 = SUH(WS1 TO WS5)/ 

VECTOR SW . SWl TO SW32 

VECTOR SX = SXl TO SX32 

VECTOR SS = SSI TO SS5 

COMPUTE XBAR = SWX/SWT 

COMPUTE XVAR = 0 

LOOP #1=1 TO 32 

. COMPUTE #DIFF = SX(#I)/SW(#I) - XBAR 
. COMPUTE XVAR = XVAR + #01 FF * IDIFF 
END LOOP 
LOOP #1=1 TO 5 

. COMPUTE SS(#I) = SS(#I)/SWT 
END LOOP 

COMPUTE SVAR = VARIANCE (SSI TO SS5) 

COMPUTE XSE = SQRT(XVAR+SVAR) 

PRINT FORMATS XVAR, SVAR, XSE (F8.4) 

LI ST VARIABLES=SEX , UWN , SWT , XBAR , XVAR , SVAR , XSE 

FINISH 
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Table VI. 11 



(continued) 



SAS 

DATA A; 

SET STUDENT; 

ARRAY WT JKWT01-JKWT32; 

ARRAY WX WX1-WX32; 

ARRAY VALUE REDVAL1-REDVAL2 ; 

ARRAY WS WS1-WS5; 

IF (REDVALl NE .); 

WTX » WEIGHT*REDVAL1; 

DO OVER WT; 

WX = WT*REDVAL1; 
END; 

DO OVER WS; 

WS = SCALE*WEIGHT; 
END DO; 
PROC SUMMARY; 
CLASS SEX; 

VAR WEIGHT JKWT01-JKWT32 WTX WX1-WX32; 
OUTPUT 0UT=B N(WEIGHT)=UWN 

SUM(WEIGHT WTX JKWT01-JKWT32 WX1-WX32 WS1-WS5= 
SWT SW1-SW32 SWX SX1-SX32 SS1-SS5; 

DATA C; 
SET B; 

ARRAY SW SW1-SW32; 
ARRAY SX SX1-SX32; 
ARRAY SS SS1-SS5; 
XBAR = SWX/ SWT; 
XVAR = 0; 
DO OVER SW; 

DIFF = (SX/SW)-XBAR; 

XVAR = XVAR+DIFF*DIFF; 
END; 

DO OVER SS; 

SS = SS/SWT; 
END DO; 

SVAR = VAR(SS1,SS2,SS3,SS4,SS5); 
XSE = SQRT(XVAR+SVAR); 
PROC PRINT; 

VAR SEX UWN SWT XBAR X\'AR SVAR XSE; 
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Table VI. 12 

Standard Error Computation; Single-Weight Method 



SPSS-X 

GET FILE=>STUDENT/ 

KEEP.SEX , JKPAIR , JKREPL , WEIGHT , X 
VECTOR VT(32) 
VECTOR WX(32) 
SELECT IF (NOT SYSMIS(X)) 
LOOP #1-1 TO 32 
. COMPUTE WT(#I) = $SYSMIS 
. COMPUTE WX(#I) = $SYSMIS 
END LOOP 

COMPUTE WTX=WEIGHT*X 
COMPUTE #W = WEIGHT 
IF (JKREPL EQ 2) #W = -WEIGHT 
COMPUTE WT(JKPAIR) = #W 
COMPUTE VX(JKPAIR) = #W * X 

AGGREGATE OUTFILE=.*/BREAK=SEX/UWN=N( WEIGHT ) / 

SWT,SW1 TO SW32 = SUM(WEIGHT, JKWTOl TO JKWT32)/ 
SWX,SX1 TO SX32 = SUM(WTX,WXl TO WX32)/ 
VECTOR SW = SWl TO SW32 
VECTOR SX = SXl TO SX32 
COMPUTE XBAR = SWX/SWT 
COMPUTE XVAR = 0 
LOOP #1=1 TO 32 

DO IF (NOT SYSMIS(SW(#I))) 

COMPUTE #DIFF = (SWX+SX(#I) )/(SWT+SW(#I) ) - XBAR 
COMPUTE XVAR = XVAR + #DIFF * #DIFF 
END IF 
END LOOP 

COMPUTE XSE = SORT (XVAR) 

PRINT FORMATS XVAR, XSE (F8.4) 

LIST VARIABLES=SEX , UWN , SWT , XBAR , XVAR , XSE 

FINISH 



ERIC 
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Table VI. 12 



(continued) 



SAS 

DATA A; 

SET STUDENT 
ARRAY WT WT1-WT32; 
ARRAY WX WX1-WX32; 
IF (X NE .); 
WTX = WEIGHT*X; 
DO OVER WT; 

WT = .; 

WX = .; 
END; 

W = WEIGHT; 

IF JKREPL=2 THEN W = -WEIGHT; 

_I_ = JKPAIR; 

WT = W; 

WX = W*X; 

PROC SUMMARY; 

CLASS SEX; 

VAR WEIGHT WT1-WT32 WTX WX1-WX32; 
OUTPUT OUT=B N(WEIGHT)=UWN 
SUM=SWT SW1-SW32 SWX SX1-SX32; 
DATA C; 
SET B; 

ARRAY SW SW1-SW32; 
ARRAY SX SX1-SX32; 
XBAR = SWX/ SWT; 
XVAR = 0; 
DO OVER SW; 

IF SW=. THEN DO; 
SW = 0; 
SX = 0; 
END; 

DIFF = ((SWX+SX)/(SWT+SW))-XBAR; 
XVAR = XVAR+DIFF*DIFF; 
END; 

XSE = SORT (XVAR); 
PROC PRINT; 

VAR SEX IIWN SWT XBAR XVAR XSE; 
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Table VI. 13 



NAEP 1983-84 Public-Use Data Tape Description 
Grade 4/Age 9 



Data Files 


Record 
Length 


Block 
Size 


1 of 
Records 


File Name 


Data Files 


1. Student BIB Spiral Data 


1728 


19008 


26,087 


yi5RDWRl.BIB.DAT 


2. PACED TAPE 64 


641 


18589 


1,403 


yl3RDWRl.P64.DAT 


3. PACED TAPE 65 


665 


13620 


1,356 


yi5RDWRl.P65.DAT 


4. PACED TAPE 66 


649 


mil 


1,36$ 


yl5RDWRl.P66.DAT 


5. PACED TAPE 67 


644 


18676 


1,344 


yl3RDWRl.P67.DAT 


6. Excluded Student 
Questionnaire 


595 


19040 


1,416 


ifl5RDWRl.EXQ.DAT 


7. Teacher Questionnaire 


606 


16766 


1,027 


yl5RDWRl.TEQ.DAT 


8. School Questionnaire 


581 


1S592 


663 


yl5RDWRl.SCQ.DAT 


SPSS-X Control Statement Files 


9. Student BIB Spiral Data 


80 


19040 


3,028 


yi5RDWRl.BIB.SPX 


10. PACED TAPE 64 


80 


19040 


793 


yi5RDWRl.P64.SPX 


11. PACED TAPE 65 


80 


19040 


695 


yl3RDWRl.P65.SPX 


12. PACED TAPE 66 


80 


imo 


751 


yl5RDWRl.P66.SPX 


13. PACED TAPE 67 


80 


19040 


669 


yi5RDWRl.P67.SPX 


14. Excluded Student 
Questionnaire 


80 


19040 


369 


yl5RDWRl.EXQ.SPX 


15. Teacher Questionnaire 


80 


19040 


694 


yl5RDWRl.TEQ.SPX 


16. School Questionnaire 


80 


19040 


436 


yl5RDWRl.SCQ.SPX 


SAS Control Statement Files 


17. Student BIB Spiral Data 


80 


19040 


4,000 


yl5RDWRl.BIB.SAS 


18. PACED TAPE 64 


80 


19040 


947 


yl5RDWRl.P64.SAS 


19. PACED TAPE 65 


80 


19040 


892 


yi5RDWRl.P65.SAS 


20. PACED TAPE 66 


80 


19040 


916 


yi5RDWRl.P66.SAS 


21. PACED TAPE 67 


80 


19040 


854 


yi5RDWRl.P67.SAS 


22. Excluded Student 
Questionnaire 


80 


19040 


442 


yl5RDWRl.EXQ.SAS 


23. Teacher Questionnaire 


80 


19040 


1,240 


yi5RDWRl.TEQ.SAS 


24. School Questionnaire 


80 


19040 


613 


yl5RDWRl.SCQ.SAS 


Machine-Readable Catalog Files 


25. Student BIB Spiral Data 


1666 


18326 


824 


yl5RDWRl.BIB.CAT 


26. PACED TAPE 64 


1666 


18326 


195 


yl5RDWRl.P64.CAT 


27. PACED TAPE 65 


1666 


18326 


200 


yi5RDWRl.P65.CAT 


28. PACED TAPE 66 


1666 


18326 


196 


yi3RDWRl.P66.CAT 


29. PACED TAPE 67 


1666 


18326 


188 


yi5RDWRl.P67.CAT 


30. Excluded Student 
Questionnaire 


1666 


18326 


128 


yl5RDWRl.EXQ.CAT 


31. Teacher Questionnaire 


1666 


18326 


332 


yi5RDWRl.TEQ.CAT 


32. School Questionnaire 


1666 


18326 


221 


yi5RDWRl.SCQ.CAT 
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Table VI. 14 



NAEP 1983-84 Public-Use Data Tape Description 
Grade 8/Age 13 



Data Files 


Record 
Length 


Block 
Size 


# of 
Records 


File Name 


Data Files 


1. Student BIB Spiral Data 


1834 


18340 


28,465 


Y15RDWR2.BIB.DAT 


2. PACED TAPE 64 


6^^ 


18647 


1,310 


Y15RDWR2-P64.DAT 


3. FACED TAPE 65 


666 


18648 


1,276 


Y15RDWR2.P65.DAT 


4. PACED TAPE 66 


653 




1,285 


Y15RDWR2.P66.DAT 


5. PACED TAPE 67 


655 




1,289 


Y15RDWR2.P67.DAT 


6. Excluded Student 
Questionnaire 


595 


19040 


1,448 


Y15RDWR2.EXQ.DAT 


7. Teacher Questionnaire 


477 


18603 


796 


Y15RDWR2.TEQ.DAT 


8. School Questionnaire 


580 


18560 


486 


Y15RDWR2.SCQ.DAT 


SPSS-X Control Statement Files 


9. Student BIB Spiral Data 


80 


19040 


3,317 


Y15RDWR2.BIB.SPX 


10. PACED TAPE 64 


80 


19646 


794 


Y15RDWR2.P64.SPX 


11. PACED TAPE 65 


86 


19040 


699 


Y15RDWR2.P65.SPX 


12. PACED TAPE 66 




19040 


767 


Y15RDWR2.P66.SPX 


13. PACED TAPE 67 


dO 


19040 


736 


Y15RDWR2.P67.SPX 


14. Excluded Student 
Questionnaire 


80 


19040 


369 


Y15RDWR2.EXQ.SPX 


15. Teacher Questionnaire 


86 


19040 


583 


Yi5RDWR2.TEQ.SPX 


16. School Questionnaire 


80 


19040 


436 


Y15RDWR2.SCQ.SPX 


SAS Control Statement Files 


17. Student BIB Spiral Data 


80 


19040 


4,307 


Y15RDWR2.BIB.SAS 


18. PACED TAPE 64 


80 


19646 


959 


Yi5RDWR2.P64.SAS 


19. PACED TAPE 65 


86 


19040 


904 


Yi5RDWR2.P65.SAS 


20. PACED TAPE 66 


80 


19040 


943 


Y15RDWR2.P66.SAS 


21. PACED TAPE 67 


86 


19040 


926 


Y15RDWR2.P67.SAS 


22. Excluded Student 
Questionnaire 


80 


19040 


442 


Y15RDWR2.EXQ.SAS 


23. Teacher Questionnaire 


80 


19040 


1,012 


Y15RDWR2.TEQ.SAS 


24. School Questionnaire 


80 


19040 


606 


Y15RDWR2.SCQ.SAS 


Machine-Readable Catalog Files 


25. Student BIB Spiral Data 


1666 


18326 


893 


Y15RDWR2.BIB.CAT 


26. PACED TAPE 64 


1666 


18326 


197 


Y15RDWR2.P64.CAT 


27. PACED TAPE 65 


1666 


18326 


202 


Yi5RDWR2.P65.CAT 


28. PACED TAPE 66 


1666 


18326 


201 


Y15RDWR2.P66.CAT 


29. PACED TAPE 67 


1666 


18326 


200 


Y15RDWR2.P67.CAT 


30. Excluded Student 
Questionnaire 


1666 


18326 


128 


Y15RDWR2.EXQ.CAT 


31. Teacher Questionnaire 


1666 


18326 


274 


Y15RDWR2.TEQ.CAT 


32. School Questionnaire 


1666 


18326 


220 


Y15RDUR2.SCQ.CAT 
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Table VI. 15 



NAEP 1983-84 Public-Use Data Tape Description 
Grade 11/Age 17 



Data Files 


Record 
Length 


Block 
Size 


# of 
Records 


File Name 


Data Files 


1. Student BIB Spiral Data 


1989 


17901 


28,861 


Y15RDWR3.BIB.DAT 


2. PACED TAPE 64 


643 


i«t)47 


1,539 


Y15RDWR3.P64.DAT 


3. PACED TAPE 63 


666 


m64U 


1,^40 


Y15RDWR3.P65.DAT 


4. PACED TAPE 66 


653 


mil 


1,5$6 


Y15RDWR3.P66.DAT 


5. PACED TAPE 67 


655 




1,534 


Y15RDWR3.P67.DAT 


6. Excluded Student 
Questionnaire 


595 


19040 


1,361 


Y15RDWR3.EXQ.DAT 


7. Teacher Questionnaire 


47? 


18603 


9l5 


Y15RDWR3.TEQ.DAT 


8. School Questionnaire 


565 


iB7zu 


331 


Y15RDWR3.SCQ.DAT 


SPSS-X Control Statement Files 


9. Student BIB Spiral Data 


80 


19040 


3,407 


Y15RDWR3.BIB.SPX 


10. PACED TAPE 64 


80 


19040 


794 


Y15RDWR3.P64.SPX 


11. PACED TAPE 65 


80 


19040 


6^9 


Y15RDWR3.P65.SPX 


12. PACED TAPE 66 


6d 


19040 


767 


Y15RDWR3.P66.SPX 


13. PACED TAPE 67 


80 


19040 


736 


Y15RDWR3.P67.SPX 


14. Excluded Student 
Questionnaire 


80 


19040 


369 


Y15RDWR3.EXQ.SPX 


15. Teacher Questionnaire 


80 


19040 


5?i 


Y15RDWR3.TEQ.SPX 


16. School Questionnaire 


80 


19040 


448 


Y15RDWR3.SCQ.SPX 


SAS Control Statement Files 


17. Student BIB Spiral Data 


80 


19040 


4,527 


Y15RDWR3.BIB.SAS 


18. PACED TAPE 64 


80 


19040 


959 


Y15RDWR3.P64.SAS 


19. PACED TAPE 65 


66 


19040 


m 


Yl5RDWR3.P65.SAS 


20. PACED TAPE 66 


80 


19040 


943 


Y15RDWR3.P66.SAS 


21. PACED TAPE 67 


80 


19040 


926 


Y15RDWR3.P67.SAS 


22. Excluded Student 
Questionnaire 


80 


19040 


442 


Y15RDWR3.EXQ.SAS 


23. Teacher Questionnaire 


80 


19040 


1,012 


Y15RDWR3.TEQ.SAS 


24. School Questionnaire 


80 


19040 


626 


Y15RDWR3.SCQ.SAS 


Machine-Readable Catalog Files 


25. Student BIB Spiral Data 


1666 


18326 


959 


Y15RDWR3.BIB.CAT 


26. PACED TAPE 64 


1666 


18326 


197 


Y15RDWR3.P64.CAT 


27. PACED TAPE 65 


1666 


18326 


202 


Y15RDWR3.P65.CAT 


28. PACED TAPE 66 


1666 


18326 


201 


Y15RDWR3.P66.CAT 


29. PACED TAPE 67 


1666 


18326 


200 


Y15RDWR3.P67.CAT 


30. Excluded Student 
Questionnaire 


1666 


18326 


128 


Y15RDWR3.EXQ.CAT 


31. Teacher Questionnaire 


1666 


18326 


274 


Y15RD\7R3.TEQ.CAT 


32. School Questionnaire 


1666 


18326 


225 


Y15RnWR3.SCQ.CAT 
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VII. Related Machine-Readable Data '-iles 
and Printed Reports 



129 




Related Machine-Readable Data Files 



Files are available for the following NAEP reading and writing assessments: 



YEAR 


AGE 


SUBJECT AREA(S) 


1979-80 


9 


Reading/Literature 


1975-80 


13 


Reading/Literature 


l"979-8u 


17 


Reading/ Literature 


1978-79 


9 


Art/Muslc/Writing 


1978-79 


13 


Art/Music/Writing 


1978-79 


17 


Art /Music/Wri ting 


197S-79 


9, 13, 17 


Writing Essay Responses 


1977 


Adults 


Science/Energy/Reading/Health 


1974-75 


9 


Reading 


1974-75 


13 


Reading 


1974-75 


17 


Reading 


1973-74 


9 


Career and Occupational Development/Writing 


1973-74 


13 


Career and Occupational Development/Writing 


1973-74 


17 


Career and Occupational Development/Writing 


1973-74 


Adults 


Career and Occupational Development/Writing 


1973-74 


9, 13, 17 


Writing Essay Responses 


1970-71 


9 


Reading/Literature 


1970-71 


13 


Reading/Literature 


1970-71 


17 


Reading/Literature 


1970-71 


Adults 


Reading/Literature 


1969-70 


9, 13, 17 


Writing Essay Responses 



For more information alout NAEP Public-Use Data 
Tapes, contact: 



Norma Norris 

Senior Research Data Analyst 

National Assessment of Educational Progress 

22-T 

Educational Testing Service 
Princeton, NJ 08541 
(609) 734-5898 



ERIC 
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Related Printed Reports 



The following reports provide additional information concerning reading 
and writing assessments* Please note that direct sources are included for 
each publication, but that most may be obtained through the Educational 
Resources Information Center (ERIC) database: 



Applebee, A* N., Langer, J» A», & Mullis, I. V. The Writing 
Report Card: Writing Achievement in American Schools, 
1984 ♦ National Assessment of Educational Progress, 
1986(a). Available from NAEP, Educational Testing 
Service, Princeton, NJ 08541. 

Applebee, A. N., Langer, J. A., & Mullis, I. V. Writing: 

Trends Across the Decade, 1974-84 . National Assessment 
of Educational Progress, Report No. ISBN-0-88685-042-8; 
NAEP-15-W.01, 1986(b). Available from NAEP, Educational 
Testing Service, Princeton, NJ 08541 • 

Beaton, A. E. National Asessment of Educational Progress 
1983-84: FTechnical Report . National Assessment of 
Educational Progress, 1986. Available from NAEP, 
Educational Testing Service, Princeton, NJ 08541. 

DeCrow, R. Adult Reading Abilities: Definitions ar l 
Measurements . National Reading Center Foundatfon , 
Washington, DC, 6p., July 1972. 

Exercise Administrator Administrative Instructions for Age 
Classes 1, 2 & 3 Year 11, October 1979-May 1980 . 
Research Triangle Inst., Report No. NAEP-ll-EA-46, 34p., 
Durham, NC, June 1979. 

The First Assessment of Reading, 1970-71 Assessment. Released 
Exercise Set. No. 02-R-25 T Education Commission of the 
States, Denver, CO. Available from NAEP, Educational 
Testing Service, Princeton, NJ 08541. 

Gadway, C. J. Critical Reading: Theme 8, Reading . 

Education Commission of the States, Denver, CO. National 
Assessment of Educational Progress, Report No. 
NARP-02-R-08, 182p., May 1973. Available from 
Superintendent of Documents, U. S. Government Printing 
Office, Washington, DC 20402. 

Gadway, C. J. Drawing Inferences: Theme 7, Reading . 
Education Commission of the States, Denver, COT 
National Assessment of Educational Progress, Report No. 
NAEP-02-R-07, 275p., August 1973. Available from 
Superintendent of Documents, U. S. Government Printing 
Office, Washington, DC 20402. 
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Gadvay, C» J» Gleaning Significant Facts from Passages; 

Theme 5, Reading s Education Commission of the States , 
Denver, CO* National Assessment of Educational 
Progress, Report No* NAEP-02-R-05, 225p*, May 1973. 
Available from Superintendent of Documents, U. S. 
Government Printing Office, Washington, DC 20402. 

Gadvay, C. J. Main Ideas and Organization; Theme 6, 

Reading. Education Commission of the States, Denver, 
CO. National Assessment of Educational Progress, Report 
No. NAEP-02-R-06, 152p., July 1973. Available from 
Superintendent of Documents, U. S. Government Printing 
Office, Washington, DC 20402. 

Gadvay, C. J. Reading; Released Exercises . Education 
Commission of the States, Denver, CO. National 
Assessment of Educational Progress, Washington, DC, 
Report No. NAEP-02-R-20, 424p., July 1973. Available 
from Superintendent of Documents, U. S. Government 
Printing Office, Washington, DC 20402. 
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NONDISCLOSURE AGREEMENT 
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analyses and conclusions of your study. In any published articles 
or reports, you will not use NAEP's name in such a way as to imply 
participation in, or responsibility for, your study, other than an 
acknowledgment that NAEP supplied the data. 

Please sign both copies of this form and return one copy to 
Norma Norris at the address below. 



Name: 

(Please print or type) 
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Address: 



Signature: 
Date: 



Return to: Norma Norris 

Senior Research Data Analyst 
22-T 
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IX, Appendices 



Appendix 1 contains four IRT item parameter tables 
(see Chapter IV, Section F for information concerning 
reading item analysis and scale derivation). The first 
table (pages 1-4) lists each BIB/UBIB sample item used for 
IRT, its corresponding IRT parameters (A, B and C) and 
standard errors (SE), the block in which the item appears 
for each grade/age group (BLOCK), and the order in which 
the item appears within the block (ITEM), 

The second, third and fourth tables (pages 5-7) list, 
by age group, each Paced sample item used for IRT, its 
corresponding IRT parameters (A, B, and C) and standard 
errors (SE), the block in which the item appears for that 
age group (BLOCK), and the order in which the item appears 
within the block (ITEM), 

The IRT parameters A, B and C are available for 
computer extraction in the Machine-Readable Catalog (see 
Table VI, 3 in Chapter VI), 

Appendix 2 lists each cognitive and non-cognitive 
assessment item (in item number order), its corresponding 
short name (SHORT LABEL), the block(s) in which the item 
appears for each age group (BLOCK) and the order in which 
the item appears within the block(s) (ITEM), 
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M»»tDXX 1 
IRT PARMOTCRS 



5/23/86 PAOB 1 
R A E P 



BXB SMIPU: ALL QRKXS/hQZS 



FZILD 


A 


SI 


NOOllOl 


0.344 


(0.048) 


M001201 


0.712 


(0.183) 


21001202 


1.276 


(0.187) 


N001301 


0.986 


(0.118) 


N001302 


0.720 


(0*087) 


N001303 


1.534 


(0.133) 


M001401 


0.999 


(0.094) 


nooisci 


1.808 


(0*130) 


11001502 


1.643 


(0.098) 


11001503 


1.345 


(0.088) 


11001504 


1.448 


(0.089) 


11001509 


0.654 


(0.043) 


11001601 


0.622 


(0.041) 


11001602 


1.263 


(0.079) 


R001603 


0.816 


(0.073) 


M01604 


1.375 


(0.101) 


N001701 


0.981 


(0.066) 


11001702 


0.541 


(0*116) 


N001703 


1.081 


(0*080) 


11001802 


1.592 


(0*140) 


11001901 


1.644 


(0.111) 


N001906 


0.934 


(0*038) 


11002001 


1.197 


(0.065) 


N002002 


1.444 


(0.084) 


11002003 


1.583 


(0.093) 


11002101 


0.941 


(0.094) 


11002102 


1.495 


(0.100) 


lf00220^ 


1.704 


(0.118) 


11002202 


1.358 


(0*120) 


NC32203 


0.783 


(0.066) 


11002401 


1.449 


(0*096) 


11002501 


0.550 


(0*053) 


11002701 


1.024 


(0.102) 


N002702 


1.148 


(0.077) 


11002801 


1.921 


(0.114) 


11002802 


1.896 


(0.110) 


N002806 


0.331 


(0.028) 


11002902 


0.55P 


(0.050) 


11002903 


2.313 


(0.180) 


11002904 


1.289 


(0.095) 


N002905 


0.758 


(0.058) 


11002906 


1.964 


(0.148) 


11003001 


1.293 


(0.109) 


11003002 


0.309 


(0.029) 


N003003 


2.294 


(0.109) 


11003101 


1.571 


(0.100) 


11003102 


1.530 


(0.083) 


11003106 


0.704 


(0.042) 


19003201 


:.207 


(0.088) 


N003202 


1.590 


(0.124) 


19003203 


1.215 


(0.101) 


N003204 


1.457 


(0.120) 


19003301 


1.141 


(0.081) 


N003401 


1.467 


(0.150) 


I90C3501 


0.751 


(0.062) 


N003601 


1.452 


(0.116) 


N003602 


1.320 


(0.109) 


N003701 


0.736 


(0.061) 


19003702 


1.071 


(0.084) 


N003706 


0.689 


(0.036) 


N003801 


0.891 


(0.112) 


19002802 


0.414 


(0.030) 


N003803 


0.757 


(0.093) 


N003901 


1.375 


(0.192) 


N004002 


0.615 


(0.079) 


19004101 


1.096 


(0.087* 


N004201 


1.103 


(0.071) 


N004202 


0.762 


(0.072) 


19004301 


1.420 


(0.125) 


N004305 


0.621 


(0.043) 




SE 

(0.104) 
(0.403) 
(0.197) 
(0.173) 
(0.241) 
(0.131) 
(0.114) 
(0.152) 
(0.061) 
(0.086) 
(0.0f8) 
(0.144) 
(0.082} 
(0.066) 
(0.068) 
(0.063) 
(0.090) 
(0.621) 
(0.096) 
(0.131) 
(0.098) 
(0.029) 
(0.050) 
(0.055) 
(0.054) 
(0.176) 
(0.118) 
(0.078) 
(0.112) 
(0.137) 
(0.057) 
(0.100) 
(0.164) 
(0.065) 
(0.081) 
(0.092) 
(0.188) 
(0.114) 
(0.082) 
(0.087) 
(0.083) 
(0.082) 
(0.169) 
(0.065) 
(0.190) 
(0.073) 
(0.051) 
(0.124) 
(0.087) 
(0.093) 
(0.107) 
(0.112) 
(0.C78) 
(0.092) 
(0.093) 
(0.099) 
(0.097) 
(0.104) 
(0.078) 
(0.035) 
(0.251) 
(0.078) 
(0.245) 
(0.331) 
(0.214) 
(0.114) 
(0.062) 
(0.098) 
(0.131) 
(0.056) 



0.291 

0.369 

0.256 

0.400 

0.497 

0.281 

0.251 

0.225 

0.182 

0.207 

0.173 

0.0 

0.133 

0.250 

0.233 

0.269 

0.231 

0.231 

0.291 

0.217 

0.331 

0.0 

0.131 

0.203 

0.224 

0.247 

0.147 

0.200 

0.337 

0.236 

0.128 

0.205 

0.234 

0.141 

0.175 

0.143 

0.0 

0.229 

0.2'"^3 

0.197 

0.116 

0.230 

U.207 

0.168 

0.120 

0.267 

0.145 

0.0 

0.171 

0.227 

0.222 

0.238 

0.158 

0.159 

0.172 

0.203 

0.241 

0.239 

0^236 

0.0 

0.309 

0.110 

0.206 

0.232 

( .246 

0.229 

0.185 

0.291 

0.288 

0.0 



SE 

(0.053 

(0.053 

(0.037 

(0.043 

(0.097 

(0.030 

(0.055 

(0.047 

(0.026 

x0.043 

(0.032 

(0.0 

(0.046 

(0.031 

(0.033 

(0.018 

(0.059 

(0 028 

(0.u-*4 

(0.012 

(0.028 

(0.0 

(0.020 

(0.020 

(0.022 

(0.019 

(0.012 

(0.037 

(0.059 

(0.085 

(0.G23 

(0.057 

(0.032 

(0.023: 

(0.028 

(0.028 

(0.0 

(0.071: 

(0.039 

(0.041: 
(0.040: 

(0.044 

(0.013 

(0.041 

(0.006 

(0.032 

(0.023 

(0.0 

(0.056 

(0.038 

(0.039 

(0.035 

(0.049 

(0.047 

(0.061 

(0.060 

(0.048 

(0.060 

(0.032 

(0.0 

(0.018 

(0.047 

(0.019 

(0.089 

(0.093 

(0.054 

(0.024 

(0.038 

(0.032 

(0.0 



ME 9 
BliOCK ITEM 

BH 5 



AflE 13 
bLOCK ITCH 



AQE 17 
BtOCK ITEM 



BK 
BK 
BK 
BH 
BU 
BJ 
BJ 
BJ 
BJ 



BK 
BK 
BK 
BK 
BK 



BL 
DL 
BL 
BL 



BM 
BM 
BM 
BM 
BM 
BM 



BN 
BN 
BN 
BO 
BO 
BO 



BO 
BO 
BO 



10 
11 
12 
13 
15 
12 
13 
14 
15 



BJ 20 



9 
10 
11 
18 
19 



BL 22 



20 
24 
25 
26 



10 
11 
12 
14 
15 
16 



23 
24 
25 
12 
13 
14 



17 
18 
19 



BK 
BK 
BK 
BK 
BK 
BK 
BK 
BK 
BK 
BK 
BK 
BK 
BJ 
BJ 
BJ 
BJ 
BJ 
BJ 
BJ 
BJ 
BJ 
BJ 
BK 
BK 
BK 
BK 
BK 
BK 
BK 
BK 
BL 
BL 
BL 

BL 
BL 
BL 
BM 
BM 
BM 
BM 
BM 

r^i 

BM 
BM 
BM 
BM 
BM 
BN 
BN 
BN 
BN 
BN 
BH 
BN 
BN 
BN 
BN 
BN 
BN 
BO 
BO 
BO 
BO 
BO 
BO 
BO 
BO 
BO 
BO 



6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
18 
11 
12 
13 
14 
17 
18 
19 
21 
22 
24 
9 
10 
11 
12 
13 
14 
15 
16 
22 
23 
24 

25 
26 
27 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
12 
13 
14 
16 
15 
17 
18 
19 
20 
21 



BK 
BK 
BH 
BH 
BH 
BK 
BH 
BK 
BK 



BJ 
BJ 
BJ 

BJ 
BJ 
BK 
BK 
BK 
BK 
BK 
BK 
BK 
BK 

BL 
BL 
BL 
BL 
BL 
BL 
BM 
BM 
BM 
BM 
BM 
BM 
BM 
BM 

m 

BH 
BM 
BN 
BN 
BN 
BI^ 
BN 

BN 
BN 
BN 
BN 
BN 
BN 
30 
BO 
BO 



BO 
BO 
BO 
BO 



10 
11 
12 
13 
14 
15 
16 
17 
19 



12 
13 
14 

35 
17 
9 
10 
11 
12 
13 
14 
15 
16 

27 
28 
29 
30 
31 
32 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
21 
22 
23 
24 
25 

27 
28 
29 
30 
31 
32 
12 
13 
14 



21 
22 
23 
24 



144 



APPENDIX 1 
nST PARAMKTERS 

BZB SMPU: ALL QRADES/ASSS 



5/23/86 PAO^ 
17 A E P 



FZBXO) 

11004401 
11004402 
11004403 

17004501 

11004502 

11004601 

11004602 

lfOO4603 

!f004607 

1IOO4701 

19004702 

1IOO4703 

N004801 

19004901 

11005001 

R005002 

19005003 

19005101 

19005201 

19005202 

M005203 

19005301 

19005302 

19005303 

19005304 

19005305 

N005403 

19005404 

N005405 

19005406 

19005407 

19005503 

19005504 

19005505 

N005601 

19005602 

N005603 

19007301 

19007302 

N007303 

N00730i 

N0073C5 

19007306 

N007401 

N007402 

19007403 

19007404 

19007405 

19007410 

19008201 

19008202 

19008203 

19008204 

19008205 

19008210 

N008601 

N008602 

N008603 

N008701 

N008801 

N008901 

N008902 

N008907 

N009001 

N009002 

N009003 

N009004 

N009101 

N009201 

N009401 



1.694 
0.764 
1.021 



1.993 
0.859 



0.718 
1.316 



A SE 

1.718 (0.127) 
0.876 (0.075) 
1.642 (0.128) 
0.974 (0.103) 
0.680 (0.054) 
0.899 (0.078) 
1.318 (0.103) 
1.485 (0.113) 
0.797 (0.049) 
(0.101) 
(0.065) 
(0.065) 
1.257 (0.085) 
0.916 (0.057) 
(0.102) 
(0.108) 
0.737 (0.105) 
0.842 (0.061) 
0.674 (0.107) 
0.600 (0.071) 
1.143 (0.121) 
1.133 (0.146) 
1.406 (0.145) 
0.867 (0.195) 
1.810 (0.197) 
1.086 (0.121) 
1.347 (0.153) 
1.455 (0.138) 
2.018 (0.195) 
1.210 (0.116) 
1.777 (0.201) 
(0.074) 
(0.112) 
1.126 (0.092) 
1.387 (0.151) 
1.715 (0.187) 
1.487 (0.171) 
1.183 (0.091) 
0.818 (0.059) 
1.110 (0.077) 
0.887 (0.072) 
0.529 (0.042) 
1.009 (0.057) 
1.098 (0.076) 
1.304 (0.084) 
1.756 (0.119) 
0.985 (0.072) 
0.887 (0.102) 
0.851 (0.042) 
2.724 (0.302) 
1.146 (0.102) 
1.543 (0.141) 
2.600 (0.236) 
2.145 (0.188) 
0.598 (0.060) 
1.789 (0.179) 
1.368 (0.179) 
1.206 (0.118) 
1.192 (0.134) 
(0.100) 
(0.106) 
1.258 (0.102) 
0.673 (0.064) 
1.328 (0.152) 
1.177 (0.163) 
0.844 (0.203) 
1.768 (0.225) 
1.007 (0.120) 
1.795 (0.172) 
1.882 (0.127) 



1.489 
1.328 



B SB 

-1.774 (0.202) 
-0.220 (0.066) 
-1.467 (0.170) 
0.493 (0.151) 
-0.824 (0.105) 
0.179 (0.104) 
-O.085 (0.092) 
-0.516 (0.089) 
-0.617 (0.051) 
-0.515 (0.059) 
-0.928 (0.105) 
-0.651 (0.062) 
-1.258 (0.108) 
0.221 (0.060) 
1.380 (0.159) 
1.288 (0.240) 
1.905 (0.331) 
-2.140 (0.178) 
0.636 (0.230) 
0.582 (0.152) 
1.837 (0.284) 
-0.028 (0.132) 
0.387 (0.119) 
1.008 (0.344) 
0.052 (0.114) 
-0.677 (0.130) 
-0.335 (0.115) 
-1.037 (0.144) 
0.068 (0.100) 
-0.398 (0.094) 
-0.246 (0.110) 
0.356 (0.127) 
0.778 (0.147) 
-0.913 (0.121) 
-0.653 (0.125) 
0.297 (0.133) 
-0.177 (0.113) 
-0.394 (0.100) 
0.285 (0.084) 
-0.024 (0.084) 
-0.007 (0.100) 
0.010 (0.077) 
-0.116 (0.059) 
0.531 (0.096) 
-0.317 (0.07E) 
0.214 (0.093) 
0.060 (0.088) 
1.401 (0.229) 
0.671 (0.049) 
-0.471 (0.131) 
-0.065 (0.102) 
-0.289 (0.104) 
-0.228 (0.092) 
-0.256 (0.092) 
2.259 (0.237) 
-0.972 (0.171) 
-0.554 (0.122) 
-0.985 (0.137) 
-2.390 (0.342) 
-1.789 (0.173) 
-1.244 (0.138) 
-1.271 (0.140) 
-2.509 (0.256) 
-0.433 (0.097) 
-0.093 (0.087) 
0.755 (0.242) 
-0.350 (0.109) 
-1.451 (0.210) 
-1.377 (0.216) 
-1.402 (0.172) 



SB 



AGE 9 
BUDCK ITEM 



AGE 13 
BLOCK ITEM 



AGl 17 
BLOCK ITEM 





Or 


/ 


BP 


7 








oJr 


8 


BP 


8 








air 


9 


BP 


9 






0 305 (0 043) 






BP 


10 


BP 


20 








BP 


11 


BP 


21 


0 184 (0 048) 






BP 


12 


BP 


22 


0 249 in n44) 






BP 


13 


BP 


23 


0 226 (0 054) 






BP 


14 


BP 


24 








BP 


15 


BP 


25 


A JfiA f A A^Y 1 


BQ 


10 


BQ 
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0 237 in 0^7) 


im 

BSJ 


11 


BQ 


8 






0 133 in 031 ) 




12 


BQ 


9 






d 747 in 047) 


BQ 


Ij 


BQ 


10 






0 1 QO in 071 ) 


BQ 


14 


BQ 


11 


BQ 


10 


0 J^^ in oiii 






BQ 


13 


BQ 


7 


0 JtiA in 0701 






BQ 


14 


BQ 


8 


0 1 35 f 0 07J 1 






BQ 


15 


BQ 


9 


0 73C fO 0R3I 


BQ 


15 


BQ 


12 




0 4ftl in 054) 






BQ 


16 


BQ 


11 


0 70fi fO 05R) 






BQ 


17 


BQ 


12 


0 300 fo om) 






BQ 


18 


BQ 


13 


0 7 ft 3 fO n^oi 






BQ 


19 




0 1 70 f 0 030 1 






BQ 


20 






n 330 fO O^lll 






BQ 


21 






0 777 fO 03ftl 






BQ 


22 






0 777 in 077) 






BQ 


23 






0 7BQ in Oftl ) 






BR 


7 






0 1117 fO 0A7) 






BR 


8 






0 706 fO 031) 






BR 


9 






0 185 fO 054) 






BP 


10 






0 37fi fO 04Q) 






BR 


11 






0 211 fO 054) 






P2l 


14 


BR 


14 


0 710 fO 074) 






BR 


15 


BR 


15 


0 247 fO 07Q) 






BR 


16 


BR 


16 


0.253 fO 071) 






BR 


17 






0.208 fO 031) 






BR 


18 






0 306 fO 051) 






BR 


19 






0 278 (0 059) 






BU 


19 


BU 


19 


0 136 fO 03Q) 






BU 


20 


B'J 


20 


0 196 fO 043) 






BU 


21 


BU 


21 


0.223 (0 053) 






BU 


22 


BU 


22 


0.133 (0.050) 






BU 


23 


nri 

DU 




0.103 (0.035) 






BU 


24 


BU 


Z4 


0.123 (0.027) 






BU 


25 


BU 


25 


0.176 (0.045) 






BU 


26 


BU 


26 


0.233 (0.027) 






BU 


27 


BU 


27 


0.181 (0.044) 






BU 


28 


BU 


28 


0.187 (0.025) 






BU 


29 


BU 


29 


0.0 (0.0 ) 






BU 


31 


BU 


31 


0.323 (0.052) 






BY 


4 


BY 


6 


0.188 (0.052) 






BY 


5 


BY 


7 


0.247 (0.054) 






BY 


6 


BY 


8 


0.209 (0.038) 






BY 


7 


BY 


9 


0.205 (0.042) 






BY 


8 


BY 


10 


0.0 (0.0 ) 






BY 


10 


BY 


12 



(0.037) 
0.261 (0.041) 
0.140 (0.043) 
0.240 (0.088) 
(0.056) 
(0.041) 
(0.043) 
(0.0 ) 
(0.031) 



0.194 

0.148 

0.156 

0.0 

0.154 



0.178 (0.030) 
0.226 (0.032) 
(0.027) 
(0.076) 
(0.054) 
(0.036) 



0.240 
0.256 
0.301 
0.105 



BH 
BH 
BH 
BH 
BJ 
BJ 
BJ 
BJ 
BK 
BK 
BK 
BK 
BK 
BK 
BL 



6 
7 
8 
9 
18 
21 
22 
24 
12 
13 
14 
15 
16 
17 
23 
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Appmozx 1 

m PARMOTSRS 



5/23/86 PA3E 3 
N A E P 



8ZB SANPUCt ALL QBADBS/AOSS 



PZILD 


A 


SE 


B 


SE 


c 


SB 


11009601 


1.360 


(0.106) 


-1.872 


(0.207) 


0.130 


(0.053) 


11009701 


1.082 


(0.124) 


-0.654 


(0.112) 


0.164 


(0.041) 


11009702 


1.959 


(0.227) 


-0.533 


(0.131) 


0.249 


(0.028) 


11009703 


1.449 


(0.211) 


-0.165 


(0.097) 


0.258 


(0.029) 


M009704 


l.lSO 


(0.185) 


0.033 


(0.096) 


0.209 


(0.031) 


11009705 


1.957 


(0.207) 


-0.702 


(0.147) 


0.211 


(0.029) 


N009801 


1.396 


(0.134) 


-2.227 


(0.296) 


0.259 


(0.086) 


11009901 


0.976 


(0.117) 


-1.049 


(0.160) 


0.206 


(0.059) 


11010002 


1.290 


(0.137) 


-1.094 


(0.165) 


0.172 


(0.047) 


11010003 


1.657 


(0.194) 


-0.940 


(0.179) 


0.241 


(0.042) 


11010102 


1.124 


(0.193) 


-0.050 


(0.111) 


0.267 


(0.037) 


11010103 


1.795 


(0.200) 


-1.075 


(0.207) 


0.209 


(0.042) 


11010201 


1.243 


(0.121) 


-1.932 


(0.245) 


0.244 


(0.078) 


11010301 


C.702 


(0.085) 


-2.383 


(0.318) 


0.248 


(0.093) 


11010401 


0.715 


(0.087) 


-1.487 


(0.209) 


0.219 


(0.077) 


11010402 


0.928 


(0.171) 


0.132 


(0.113) 


0.222 


(0.037) 


M010403 


1.031 


(0.197) 


0.465 


(0.153) 


0.190 


(0.027) 


N010501 


2.023 


(0.139) 


-1.490 


(0.190) 


0.204 


(0.046) 


N010502 


1.204 


(0.114) 


-1.196 


(0.154) 


0.156 


(0.049) 


N010503 


1.455 


(0.123) 


-1.460 


(0.184) 


0.159 


(0.048) 


N010504 


2.300 


(0.166) 


-1.114 


(0.174) 


0.182 


(0.032) 


N010601 


1.604 


(0.196) 


-0.634 


(0.136) 


0.246 


(0.036) 


11010602 


1.788 


(0.^:4) 


0.309 


(0.153) 


0.306 


(0.023) 


N010603 


1.359 


(0.199) 


-0.258 


(0.105) 


0.234 


(0.033) 


N010604 


1.637 


(0.250) 


-0.101 


(0.106) 


0.235 


(0.026) 


N010605 


1.220 


(0.190) 


-0.069 


(0.096) 


0.184 


(0.031) 


N010701 


1.064 


(0.122) 


-1.283 


(0.188) 


0.179 


(0.063) 


N010801 


1.084 


(0.119) 


-0.471 


(0.087) 


0.260 


(0.035) 


11010902 


1.564 


(0.153) 


-0.467 


(0.087) 


0.241 


(0.026) 


N010903 


1.850 


(0.157) 


-C.564 


(0.096r 


0.193 


(0.022) 


N010904 


1.522 


(0.170) 


-0.245 


(0.080) 


0.275 


(0.024) 


KOllOOl 


1.279 


(0.116) 


-0.879 


(0.113) 


0.228 


(0.037) 


170X1002 


1.657 


(0.167) 


-0.315 


(0.081) 


0.252 


(0.022) 


M011003 


2.<lu 


(0.169) 


-0.928 


(0.147) 


0.241 


(0.024) 


N011004 


1.788 


(0.159) 


-0.543 


(0.097) 


0.226 


(0.023) 


NOlllOl 


1.568 


(0.141) 


-0.541 


(0.090) 


0.197 


(0.025) 


N0\1201 


0.911 


(0.117) 


-0.259 


(0.085) 


0.260 


(0.037) 


N011301 


1.653 


(0.141) 


-C.756 


(0.111) 


0.211 


(0.028) 


N011302 


0.992 


(0.119) 


-0.430 


(0.089) 


0.227 


(0.039) 


N011401 


0.838 


(0.194) 


0.697 


(0.227) 


0.334 


(0.030) 


N011402 


0.822 


(0.1:^9) 


0.010 


(0.102) 


0.288 


(0.041) 


N011403 


0.971 


(0.192) 


0.621 


(0.186) 


0.270 


(0.025) 


N011404 


1.327 


(o.as) 


0.492 


(0.151) 


0.220 


(0.019) 


N012901 


1.091 


(0.091) 


-1.601 


(0.169) 


0.136 


(0.053) 


N013001 


1.020 


(0.122) 


-0.343 


(O.Of -) 


0.165 


(0.036) 


N013002 


0.972 


(0.121) 


-0.38i 


(O.09C) 


0,187 


(0.040) 


N013003 


1,717 


(0.164) 


-1.123 


(0.172) 


0.234 


[0.041) 


NC13004 


0.994 


(0.115^ 


-0.946 


(0.138) 


0.216 


[0.056) 


N013101 


1.757 


(0.136) 


-1.560 


(0.196) 


0.215 


;0.052) 


N013102 


1.401 


(0.146) 


-0.789 


(0.124) 


0.218 


(0.039) 


N013103 


0.954 


(0.097) 


-0.868 


(0.117? 


0.147 


0.046) 


N013104 


0.760 


(0.116) 


-0.421 


(0.111) 


0.216 


:0.055) 


N013201 


1.665 


(0,211) 


-0.693 


(0.160) 


0.181 


0.037) 


N013301 


1.232 


(0.161) 


-1.557 


(0.268) 


0.253 


0.077) 


N013401 


1.2U3 


10.177) 


-0.250 


(0.107) 


0.157 


0.035) 


N013402 


1.433 


(0.189) 


-0.862 


(0.175) 


0.205 


0.048) 


N013403 


1.494 


(0.223) 


-0.278 


(0.116) 


0.199 


0.033) 


N014001 


1.238 


(0.153) 


-0.857 


(0.149) 


0.249 


0.048) 


N014101 


0.758 


(0.071) 


-1.283 


(0.142) 


0.169 


0.059) 


N014201 


1.207 


(0.134) 


-1.218 


(0.169) 


0.136 


0.052) 


N014301 


1.755 


(0.191) 


-0.82J 


(0.158) 


0.190 


0.035) 


N014302 


1.074 


(0.136) 


-0.496 


(0.108) 


0.181 


0.041) 


N014303 


1.721 


(0.187) 


-1.041 


(0.188) 


0.208 


0.041) 


N014501 


0.432 


(0.065) 


-2.264 


(0.348) 


0.0 ( 


0.0 ) 


N014502 


0.934 


(0.123) 


-2.664 


(0.406) 


0.0 ( 


0.0 ) 


N014503 


0.624 


(0.133) 


-4.120 


(0.903) 


0.0 ( 


0.0 ) 


N015101 


0.d32 


(0.110) 


0.343 


(0.168) 


0.234 ( 


0.067) 


N015102 


2.533 


(0.236) 


0.548 


(0.207) 


0.216 ( 


0.030) 


N015103 


2.401 


(0.200) 


0.660 


(0.197) 


0.219 1 


0.028) 


N015104 


1.707 


(0.184) 


0.441 


(0.193) 


0.276 ( 


0.045) 



AGE 9 AGE 13 AlQE 17 

BU)CK ITEM BIX>CK ZTEM BLOCK ITEM 



nr. 
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5 


BM 


g 


CM 


7 


BM 
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BN 
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BO 
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BO 


20 


BO 
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BO 
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BP 
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BP 
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BP 
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BP 
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BP 
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BP 
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BP 


16 


BP 
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BP 
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BP 
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BO 
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BO 
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no 
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BR 


5 


BR 




BR 


7 


BR 
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BR 
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BR 


10 


BR 


11 


BR 


12 


BR 


13 


BR 


14 


BR 


15 


BR 


16 


BU 


19 


BU 


20 


BU 


21 


BU 


22 


BU 


23 


BU 


24 


BU 


25 


BU 


26 


BU 


27 


BV 


29 


BV 


30 


BV 


31 


BV 


32 


BV 


33 


BM 


13 


BQ 


21 


BV 


34 


BN 
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BN 
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BN 


16 


BV 


35 


BV 


35 


BV 
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AmMDrx 1 

XIT PARMOTSItS 



8XB SAMPLE: ALL QRADES/AOES 



5/23/86 PAOB 
N A E P 



PXXLD 

11015201 
1V015502 
W>15303 
11015504 
1IO15505 
110X5901 
11015902 
If0l5903 
11015907 
N016001 
11016002 
R016003 
t^l6004 
1IO16005 
11016006 
N017001 
N017002 
11017003 



1.089 
1.273 
0.912 
1.189 

0. 683 
1.021 
1.380 

1. X82 
0.657 
1.043 
1.386 
0.906 
1.095 
1.734 
1.357 
1.518 
1.935 
1.833 



SE 

(0.126) 
(0.126) 
(0.119) 
(0.121) 
(0.083) 
(0.133) 
(0.165) 
(0.129) 
(0.062) 
(0.122) 
(0.154) 
(0.103) 
(0.126) 
(0.175) 
(0.137) 
(0.157) 
(0.138) 
(0.129) 



-^.766 
0.189 
0.756 
0.110 

-^.175 
0.371 
0.726 
1.101 
0.858 
0.033 
1.247 
0.354 
0.103 
0.156 
0.424 
0.484 
1.100 
1.770 



SE 

(0.150) 
(0.140) 
(0.216) 
(0.138) 
(0.146) 
(0.204) 
(0.234) 
(0.224) 
(0.106) 
(0.164) 
(0.276) 
(0.157) 
(0.165) 
(0.152) 
(0.161) 
(0.175) 
(0.193) 
(0.248) 



0.227 

0.209 

0.247 

0.220 

0.247 

0.333 

0.317 

0.153 

0.0 

0.285 

0.456 

0.205 

0.271 

0.230 

0.203 

0.213 

0.196 

0.177 



SE 

(0.085) 
(0.057) 
(0.056) 
(0.062) 
(0.087) 
(0.068) 
(0.043) 
(0.032) 
(0.0 ) 
tO.078) 
(0.028) 
(0.065) 
(0.074) 
(0.054) 
(0.049) 
(0.042) 
(0.022) 
(0.016) 



AGE 9 
BLOCK ITEM 



AGE 13 
BLOCK ITEM 



AGE 17 
BLOCK ITEM 



BN 


26 


BP 


16 


BP 


17 


BP 


18 


BP 


19 


BQ 


14 


BQ 


15 


BQ 


16 


BQ 


17 


DO 


15 


DO 


16 


DO 


17 


DO 


18 


DO 


19 


DO 


20 


BH 


7 


BH 


8 


BH 


9 
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AmxDzx : 

net PAMKETSRS 



5/23/86 PAGE 5 
If A B P 



PJyCI SMfPLKS: MB 9 



fztu> 


A 


SB 




MOOlSOl 


1.809 


(0.195) 


—2, 822 


II001502 


1.843 


(0.210) 


-0.493 


11001503 


1.439 


(0.104) 


-1.348 


II001504 


1.565 


(0.189) 


-0.477 


11001601 


0.811 


(0.065) 


-1.265 


19Q01602 


1.106 


(0.097) 


-0.788 


11001603 


0.691 


(0.110) 


-0.167 


11001604 


0.943 


(0.126) 


0.157 
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17 


(0.072 ) 


0 MJ 
V • X /A 


(0.019 ) 


P3 


15 


(0.055) 


0.187 


(0.039) 


P3 


16 


(0.100) 


0.066 


(0.010) 


P3 


17 


(0.2Z ) 


0.258 


(0.059) 


^1 


7 


(0.168 ) 


0.265 


(0.061) 


Pi 


Q 


(0.112 ) 


0.258 


(0.057) 


Pi 


g 


(0 . 146 ) 




(0 .061 ) 


Pi 


X V 


(0.198 ) 


0 .266 


(0.060) 


P3 


19 


(0.111 ) 


0.230 


(0.052 ) 


P2 


10 


(0.164) 


0.245 


(0.055) 


Pi 


13 


(0.078) 


0.310 


(0.039) 


Pi 


li 


(0.063) 


0.132 


(0.031) 


P4 


2 


(0.427) 


0.206 


(0.059) 


P4 


3 


(0.296 ) 


0.284 


(0.046) 


P4 


4 


(0.581 ) 


0.256 


(0.060) 


P3 


14 


(0.103) 


0.272 


(0.052) 


P4 


18 


(r.i08) 


0.260 


(0.056) 


P4 


19 


(0.131) 


0.266 


(0.056) 


P4 


16 


(0.424) 


0.260 


(0.061) 


P4 


17 


(0.297) 


0.262 


(0.061) 


P3 


20 


(0.053) 


0.225 


(0.032) 


P3 


16 


(0.153 ) 


0.228 


(0.031) 


P2 




(0.078) 


0 • 240 


(0.02 f.) 


P2 


7 


(0.116 ) 


0.126 


(0.023) 


f2 


8 


(0.995) 


0 .272 


(0.064) 


P2 


2 


(0.092) 


0.497 


(0.040) 


P4 


23 


(0.154) 


0.246 


(0.030) 


P4 


25 


(0.128) 


0.395 


(0.060) 


Pi 


1 


(0.107) 


0.427 


(0.024) 


PI 


2 


(0.079) 


0 453 


(0.024) 


PI 


3 


(0.062) 


0.196 


(0.034) 


PI 


4 


(0.061) 


0.183 


(0.036) 


PI 


5 


(0.052) 


0.128 


(0.025) 


PI 


6 


(0.110) 


0.216 


(0.033) 


P4 


14 


(0.259,' 


0.246 


(0.058) 


P3 


1 


(0.110) 


0.243 


(0.055) 


P3 


< 


(0.153) 


0.239 


(0.054) 


P3 


3 


(0.169) 


0.257 


(0.059) 


P3 


4 


(0.182) 


0.261 


(0.051) 


P3 


5 
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APPITOIX 2 5/23/66 PAGE 1 

XTBN USMC ACROSS AGES k BUXX3 N A E P 

AQB 9 AGE 13 AGE 1'' 



fltU> 


SBORT XABEL 


BLOCK ITEM 


BLOCK 


ITEM 


BLOCK 


ITEM 


lf000102 


DALI: VKQWKl TPAIT SCORE (RATER 1) 


BA 


13 


BA 


13 


BA 


13 






P2 


1 


P2 


1 


P2 


1 


N000103 


CAZiI: FRIHARY TRAIT SCORE (RATER 2) 


BA 


13 


BA 


13 


BA 


13 






P2 


1 


P2 


1 


P2 


1 


N000104 


DAZJ: FRIHARY TRAIT SCORE 


BA 


13 


BA 


13 


BA 


13 






P2 


1 


V7 


1 


P2 


1 


N000105 


DALI: PRIMMGT TRAIT: FIRST RATER ID 


BA 


13 


BA 


13 


BA 


13 






P2 


1 


P2 


1 


P2 


1^ 


11000106 


CAZJ: FRIMAR7 TRAIT: SECOlfD RATER ID 


BA 


13 


BA 


13 


BA 


13 






P2 


1 


P2 


1 


P2 


1 


N000107 


CAU: PRIMARY TRAIT: RSSOL. 'o\TER ID 


BA 


13 


BA 


13 


BA 


13 






P2 


1 


P2 


1 


P2 


1 


N000108 


tAZJ: HOLISTIC SCORE (RATER 1) 


BA 


13 


BA 


13 


BA 


13 






P2 


1 


P2 


1 


V2 


1 


11000109 


DALI: HOLISTIC SCORE (RATER 2) 


BA 


13 


BA 


13 


BA 


13 






P2 


1 


P2 


1 


P2 


1 


NOOOllO 


DALI: HOLISTIC SCORE 


BA 


13 


BA 


13 


BA 


13 






P2 


1 


P2 


1 


P2 


1 


NOOOlll 


DALI: FIRST HOLISTIC RATER ID 


BA 


13 


BA 


13 


BA 


13 






P2 


1 


P2 


1 


P2 


I 


N000112 


DALI: SECOND HOLISTIC RATER ID 


BA 


13 


BA 


13 


BA 








P2 


1 


P2 


1 


P2 


1 


11000113 


DALi: RESOIOTION HOLISTIC RATER ID 


BA 


13 


BA 


13 


BA 


13 






P2 


1 


P2 


1 


P2 


1 


N000202 


SCHOOL RULE: PRIMARY TRAIT SCORE (RATER 1) 


BB 


16 


BB 


16 


BB 


16 


N000203 


SCHOOL RULE: PRIMARY TRAIT SCORE (RATSR 2) 


BB 


16 


BB 


16 


BB 


16 


N000204 


SCHOOL RULE: PRIMARY TRAIT SCORE 


BB 


16 


BB 


16 


BB 


16 


N000205 


SCHOOL RULE: PRIMARY TRAIT: FIRST HATER ID 


BB 


16 


BB 


16 


BB 


16 


N000206 


SCHOOL RULE: PRIMARY TRAIT: SECOND RATER ID 


BB 


16 


BB 


16 


BB 


16 


N000207 


SCHOOL RULE: PRIMARY TRAIT: RESOL. RATER ID 


BB 


16 


BB 


16 


BB 


If 


N000302 


RECREATION OPP: PRIMARY TRAIT SCORE (RATER 1) 






BC 


23 


BC 


23 


N000303 


RECREATION C^P: PRIMARY TRAIT SCORE (RATER 2) 






BC 


23 


BC 


23 


N000304 


RECREATION OPP: PRIMARY TRAIT SCORE 






BC 


23 


BC 


23 


11000305 


RECREATION OPP: SECONDARY TRAIT: MADE NOTES 






BC 


23 


BC 


23 


11000306 


RECREATION OPP: PRIMARY TRAIT: FIRST RATER ID 






BC 


23 


BC 


23 


11000307 


RECREATION OPP: PRIMARY TRAIT: SECOND RATER ID 






BC 


23 


BC 


23 


N000308 


RECREATION OPP: PRIMARY TRATT* PPCrtT o^TVO rn 






BC 


23 


BC 


23 


N000402 


FOOD ON FRONTIER: PRIMARY TRAIT SCORE (RATER 1) 


BD 


25 


BD 


25 


BD 


25 


N000403 


FOOD ON FRONIIER: PRIMARY TRAIT SCORE (RATER 2) 


BD 


25 


BD 


25 


BD 


25 


N000404 


FOOD ON FRONTIER: PRIMARY TRAIT SCORE 


BD 


25 


BD 


25 


BD 


25 


N000405 


FOOD ON FRONTIER: PRIMARY TRAIT: FIRST RATER ID 


BD 


25 


BD 


25 


BD 


25 


N000406 


FOOD ON FRONTIER: PRIMARY TRAIT: SECOND RATER ID 


BD 


25 


BD 


25 


BD 


25 



ERIC 



152 



APPBIDIX 2 

ZTEH USAOI ACROSS AOXS 6 BI/X3CS 



5/23/86 PAOE 2 
N A E P 



FXSLD 


SBKiRr ZAsCL 


AOB 

BLOCK 


9 

ITEM 


AGE 13 
BLOCK ITEM 


AGE 
BLOCK 


17 
ITS 




FOOD OH PROtfnCR: FRINABY TRAIT: RCSOL. RATER ID 


BD 


25 


BD 


25 


BD 


25 




DISSECnUCI FHOOS: PRINARST TRAIT SCORE (RATER 1) 






BE 


10 






M0OOSO3 


ncOGS: FRinAKz mAIT SCORE (RATER 2) 






BE 


10 








i;i5»cuTiiKi FROCiS: FRIrlARx TRAIT SCORE 






BE 


10 








uida&v.TirR» rROOS: ncinARY TRAIT: FIRST RATER ID 






BE 


10 








DissEcrxxw FROCiS: FRInARz TRAIT: SEO^iD RATER ID 






BE 


10 






nv/Uv^u / 


DISSBCnNQ FROQS: PRII!ARY TRAIT: RESOL. RATER ID 






BE 


10 






nuuuuu* 


XYZ COMPANV: PRIMARY TRAIT SCORE (RATER 1) 


BE 


11 


BE 


11 








AX^ CtxiFAnx: FRInARY TRAIT SCORE (RATER 2) 


BE 


AX 


BE 


11 






Uwuuv 


AXA vAJflrAHx • mirlARx TRAIT SCORE 


BE 


11 


BE 


11 








(aXvARxZ rRxflARx TRAIT: FIRST RATER ID 


BE 


11 


BE 


11 








AX A uxVAnx : FKUnXz TRAIT: SEC^fD RATER ID 


BE 


11 


BE 


11 






nuuuo V / 


XY2 COMPANY: PRIMARY TRAIT: RESOL. RATER ID 


BE 


11 


BE 


11 






Mnnn*TA9 


SWXMMIIIS POOL: PRIMARY TRAIT SCORE (RATER 1) 


BF 


6 


BF 


6 


BF 


6 


nuuu tvo 


SWMQiro POOL: PRIMARY TRAIT SCORE (RATER 2) 


BF 


6 


BF 


6 


BF 


6 


nuuu /v4 


swinHUro POOL: primary trait SCORE 


BF 


6 


BF 


6 


BF 


6 




aifiminii rooL: JrRIriARY TOAIT: FIRST RATER ID 


BF 


6 


BF 


6 


BF 


6 


rivuu / w 


swimiwi POOL: PRIMARY TRAIT: SECOND RATER ID 


BF 


6 


BF 


6 


BF 


6 


Mnnn7n7 

IiUUU / u / 


SWIMMING POOL: PRIMARY TRAIT: RESOL. RATER ID 


BF 


6 


BF 


6 


BF 


6 


MUUUw V* 


PET: PRIMARY TRAIT SCORE (RATER 1) 


BF 


7 


BF 


7 






nuuuo 


PET: PRIMARY TRAIT SCORE (RATER 2) 


BF 


7 


BF 


7 








PET: PRIMARY ^lAIT SCORE 


BF 


7 


BF 


7 






nuuuovs 


PET: PRIMARY TRAIT: FIRST RATER ID 


BF 


7 


BF 


7 








PET: PRIMARY TRAIT: SECOND RATER ID 


BF 


7 


BF 


7 








rCT. FRInARY TRAIT: RESOL. RATER ID 


BF 


7 


BF 


7 






N0DQ902 


Muxu cyrAXXun. FRXnAKx TRAIT SCORE (RATER 1) 


BG 


7 


BG 


7 








luUiio STAXlOri: PRIMARY TRAIT SCORE (RATER 2) 


BG 


7 


BG 


7 






N0QQ9Q4 

4« V U U 7 U ^ 


"'WAV oliviXMl. irivjunAni TRAIT SCOR£ 


RG 


7 


BG 


7 








luu/iu oiAXiun. PRifiARy TRAIT: FIRST RATER ID 


BG 


7 


BG 


7 






M U U U 7 V O 


MAUIO STATION: PRIMARY TRAIT: SECOND RATER ID 


BG 


7 


BG 


7 








ixnuxu orATiori. rRXXlARi TRAIT: RESOL. RATER ID 


BG 


7 


BG 


7 






N001002 


APPLEBY HOUSE: PRIMARY TRAIT SCORE (RATER 1) 


BG 


8 


BG 


8 


BG 


8 


N001003 


APPLEBY HOUSE: PRIMARY TRAIT SCORE (RATER 2) 


BG 


8 


BG 


8 


BG 


8 


N001004 


APPLEBY HOUSE: PRIMARY TRAIT SCORE 


BG 


8 


BG 


8 


BG 


8 


N001005 


APPLEBY HOUSE: PRIMARY TRAIT: FIRST RATER ID 


BG 


8 


BG 


8 


BG 


8 


N001006 


APPLEBY HOUSE: PRIMARY TRAIT: SECOND RATER ID 


BG 


8 


BG 


8 


BG 


8 
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APPSHDXX 2 

ZTBM USMC ACROSS AQSS k BXOCKS 



F2EU) SHORT lABCL 

11001007 APPLm HOUSE: PRIMARY TRAIT: RESOL. RATER ID 

NOOllOl PlCITXRE:aSREAL NTTH TOY INSIDE IS PAX 

11001201 UXn DIST:RATE on CALL-Ul^fER EVENING RATS 

N001202 U)NQ DIST:PERS0N CALLS DIPF-OPR ASSISTED 

11001301 KOLA C0UF0lf:GO0D FOR ANY SIZE CARTON 

11001302 KOLA COUFON:USE Oil NOV. 10, 1970 
N001303 KOLA COUPON: PAYMENT IS 12 CENTS 
N001401 VERSE:DBCK op cards DESCRIBED IN POEM 

N001501 nuts: DEVIL PUT PEARL IN WULNUT 

K001502 NUTS: FARM VTEFE VIAS CI£VER AND PRACTICAL 

N001503 NUrS: WANTED TRICK SOMEONE INTO CRACKING WAUJUTS 

N001504 NUTS; PLAN WRONG-^^OMAN WAS TOO CLEVER FOR HIM 

N001505 NUTS: IS THIS A GOOD STORY? 

N001507 NUTS: PRIMARY TRAIT SCORE (RATER 1} 

N001508 NUTS: PRIMARY TRAIT SCORE (RATER 2) 



AGE 9 
BLOCK ITEM 



N001509 NUTS: PRIMARY TRAIT SCORE 



N001510 NUTS: SECONDARY TRAIT: CONTENT 



NO 01 511 NUTS: SECONDARY TRAIT: FORM 



N001512 NUTS: SECONDARY TRAIT: SUBJ. REACTION 



N001513 NUTS: PRIMARY TRAIT: FIRST RATER ID 



N001514 NUTS: PRIMARY TRAIT: SECOND RATER ID 



N001515 NUTS: PRIMARY TRAIT: RESOL. RATER ID 



N001601 1ST AM:BITTER WINTER-E5CTREMEL^ COLD 



N001602 1ST AM: ICE AGE PE-)PLE DEPENDED ON ANIMALS TO UVE 



N001603 1ST AM:NO LAND BRIDGE NC^COVERED WITH WATER 



5/23/86 PAGE 3 
N A E P 

AGE 13 AGE 17 

BLOCK ITEM BLOCK ITEM 



EG 



BH 
P3 



BH 10 

P3 1 

BH 11 

P3 2 

BH 12 

Po 3 

BH 13 

P3 4 

BH 14 

P3 5 

BH 15 

P3 6 

BH 15 

P3 6 

BH 15 

P3 6 

BH 15 

P3 6 

BH 15 

i>3 6 

BH 15 

P3 6 

BH 15 

P3 6 

BK 15 

P3 6 

BH 15 

P3 6 

BJ 12 

P3 e 

BJ 13 

P3 9 

BJ 14 

P3 10 



BG 8 

BK 6 

BH 7 

P3 26 

BH 8 

P3 27 

BH 9 

BH 10 

BH 11 

BH 12 

P3 21 

BH 13 

BH 14 



BH 16 



BH 17 



BH 18 



BH 16 



BH 18 



BH 18 



BH 18 



BH 18 



BH 18 



BH 18 



BJ 11 

P3 6 

BJ 12 

P3 9 

BJ 13 

P3 10 



BQ 



P3 26 

P3 27 

BH 10 

BH 11 

BH 12 

BH 13 

P3 21 

BH 14 

BH 15 



BH 15 BH 16 



BH 17 



BK 18 



BH 19 



BH 19 



BH 19 



BH 19 



BH 19 



BH 18 BK 19 



BH 19 



BH 19 



BH 19 



P3 8 



P3 9 



P3 10 



ERIC 
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APPERDZX 2 

ITEN USM3B ACPOSS AGES BLOCKS 



PZBLD SaORT ZABSL 

If001€04 1ST AMtMMN PtmPOSE-EXFUt ICS AQE SEITLERS-N. AH. 

If001€05 1ST AM:HOIf nrrSRESTXllQ HAS THIS ARTICXE 

II001606 1ST AM:HG«f HARD VAS THIS ARTICXE TO READ 

N001701 BOOR a;UB:SHIPPINS COSTS HIGHER IM CANADA 

W)01702 BOOK OJJBlSmD 190 Wtm TILL BILLED 

N001703 BOOK CLUB:BU7 6 MORE 

11001801 FL3f:HA17r OF THOUGKT-LACK OF THINKItfG 

N001802 FL^tFACING PROBLEMS SIMILAR TO HIS 0^ 

N001901 CHARLEYl: MANS FEARS 

NO01904 CHARLEYl: PRIMAI^f TRAIT SCORE (RATER 1) 

N001905 CHARLEYl: PRIMARY TRAIT SCORE (RATER 2) 

N001906 CHARLEYl: PRIMARY TRAIT SCORE 

NC01907 CHARLEYl: SECONDARY TRAIT: CONTENT 

M001908 CHARLEYl: SECONDARY TRAIT: TORM 

N001909 CHARLEYl: SECONDARY TRAIT: SUBJ. REACTION 

N001910 CHARLEYl: SECONDARY TRAIT: PIECES OF ZVlTiZuCE 

N001911 CHARLEYl: PRIMARY TRAIT: FIRST RATER ID 

N001912 CHARLEYl: PRIMARY TRAIT: SECOND RATER ID 

N00191? CHARLEYl: PRIMARY TRAIT: RESOL. RATER ID 

N002001 WISH COULD FLY:GOSSAMER CONDOR 1ST MUSCLE-PCWERED 

N002002 WISH COULD FI«Y:BIKE RACER, BRYAN ALLEN FlilW CONDOR 

N002003 WISH COULD FT^ :MACCREADY PLANE DIFF-SIMPIJR/LIGHTR 

N002101 VIRUSES: DIFFICULT TO STUDY 

NO 02102 VIRUSES: CLOCHE IDEA-GIV. PROOF TO SLTPORT 

N002201 PHONt; BILL:FSB 14 CALL FROM ATHENS, GA 

N002202 PHONE BILL:TEB 14 CALL TO ST PAUL, MN 

N002203 PHONE BILL:FEB 14 CALL COST $.75 

N002302 THE DOOR: PRIMARY TRAIT SCORE (RATER 1) 



AGE 9 
BLOCK ITEM 



BJ 
P3 



BJ 
P3 



BJ 
P3 



BJ 
P4 

BJ 
P4 



BK 
P2 

BK 
P2 

BK 
P2 

BK 
P4 

BK 
P4 



15 
11 

16 
12 

17 
13 



19 
14 



20 
15 



9 
10 

10 
11 

11 
12 

18 
5 

19 
6 



AGE 13 

BLOCK ITEM 

BJ 14 

P3 11 

BJ 15 

P3 12 

BJ 16 

P3 13 

AJ 17 

P2 3 

BJ 18 

P2 4 

BJ 19 

P2 5 

BJ 70 

P4 20 

BJ 21 

P4 21 

BJ 22 

BJ 24 

BJ 24 

BJ 24 

BJ 24 

BJ 24 

BJ 24 

BJ 24 

BJ 24 

BJ 24 

BJ 24 

BK 9 

P2 11 

BK 10 

P2 12 

BK 11 

P2 13 

BK ?2 

P4 5 

BK 13 

P4 6 

BK 14 

P2 14 

BK 15 

P2 15 

BK 16 

P2 16 

BK \1 



5/23/86 
N A B P 

AGE 

BLOCK 



PAGE 



17 

ITEM 



P3 11 



P3 12 



P3 13 



BJ 
P2 

BJ 
P2 

BJ 
P2 



BJ 

BJ 

BJ 

BJ 

BJ 

BJ 

BJ 

BJ 

BJ 

BJ 

BJ 

BK 
P2 

BK 
P2 

BK 
P2 

BK 
P4 

BK 
P4 

BK 
P2 

BK 
P2 

BK 
P2 



12 
3 

13 
4 

14 

5 



P4 20 



P4 21 



15 

17 

17 

17 

17 

17 

17 

17 

17 

17 

17 

9 
11 

10 
12 

11 

13 

12 
5 

13 

6 

14 

14 

15 
15 

16 
16 



BK 17 



ERLC 



15c 



APPSIDIX 2 

XTOI USAOK ACROSS A3£S 6 BLOCKS 










5/23/86 
N A E P 


PAGE 


PIILD 


SHOiCr LABEL 


AGE 
BLOCK 


9 

ITEM 


AGE 

BLOCK 


13 

ITEM 


AGE 
BLOCK 


17 

ITEM 


11002303 








BK 


17 


BK 


17 


N002304 


tEHB ZXX3R: PRIMAmr TJXTT QroPP 






BK 


17 


BT 


17 


11002305 


THB DOOR: gyPHRit TRAIT SCORE 1 KCOcrwrPTP 






BK 


17 


BK 


17 


N002306 


TUS DOOR: OTHEH TRAIT SCORE 2 PERSOMXf^i^MAT VTxr 






BK 


17 


BK 


17 


N002307 


THB DOOR: OTHER TRAIT SCORE 3 EMOTIONAL 






BK 


17 


BK 


17 


11002308 


TBK DOOR: OTHER TRAIT SCORE 4 SYNOPSIS 






3K 


17 


BK 


17 


11002309 


TBK DOOR: OTHER TRAIT SCORE 5 II9FERENCII96 






BK 


17 


BK 


17 


11002310 


THE DOOR: fffWltl* TRAIT SCORE £ aEMTRXT T7&TTrM 






BK 


17 


BK 


17 


M002311 


^HB DOOR: OTHER TRAIT SroRE 7 ANXTVCtc 






BK 


17 


BK 


17 


N002J12 


THE DOOR! fJff^Kit TRATT ^rnnis II omipu untiirc 






BK 


17 


BK 


17 


N002313 


THE DOOR: OTHER TRAIT SCORE 9 EVALUATION 






BK 


17 


BK 


17 


N002314 


THB DOOR: PRIMARY TRAIT: FIRST RATER ID 






BK 


17 


BK 


17 


N002315 


IHE DOOR: PRIMARY TRAIT: SEO^H) RATER ID 






BK 


17 


BK 


17 


N0C2316 


THE DOOR: PRIMARY TP&TT* PPViT pjw^tj tii 






DK 


17 


BK 


17 


N002401 


MOSQUITO: SIZE MOSQUITOES EXAGGERATED 


BL 
P2 


22 
7 


BL 


22 






N002501 


MARY: WILL GET MONEY FROM NEITHER 


P2 


13 


BL 
P2 


23 
17 


BL 
P2 


27 
17 


N002701 


ATMOSPHERE: 4 VIOROS CUE—FIRST, NEXT, ABOVE, FINALLY 






BL 


24 


BL 


28 


N002702 


ATMOSPHERE : SCIENTISTS KNOW MOST ABOUT TROPOSPHERE 


BL 


20 






BL 


29 


N002801 


BETHUNE: ROOSEVELT HONOR HER BY MAKING HER DI RECTO 


BL 


24 


BL 


25 


BL 


30 


NO02802 


BETHUNE: START HER SdoOL TO rwirnTf? nr^rv <~UTTnt>M 


BL 


25 


BL 


26 


BL 


31 


N002804 


BETHUNE: prtmapv TRATT cmpp /patst> ii 


BL 


26 


BL 


27 


BL 


32 


N002805 


BETHUNE: PRIMARY TRAIT SCORE (RATER 2) 


BL 


26 


BL 


27 


BL 


32 


17002806 


BETHUNE: PRIMARY TRAIT SCORE 


BL 


26 


BL 


27 


BL 


32 


N002807 


BETHUNE: PRIMARY TRAIT: FIRST RATER ID 


BL 


26 


BL 


27 


BL 


32 


N002808 


BETHUNE: PRIMARY TRAIT: SECOND RATER ID 


BL 


26 


BL 


27 


hu 


32 


N002809 


BETHUNE: PRIMARY TRAIT: RE SQL* RATER ID 


BL 


26 


BL 


27 


BL 


32 


N002901 


SOCCER: DID YOU T.TKP ni?ADTfia tutc APTTrrr 






BM 


5 


BM 


5 


N002902 


SOCCER: MOST POPULAR RPfTAIKP PTAVPn RV >*TTTTnMC 






BM 


6 


BM 


6 


N0U29U3 


SOCCER:KIN0 ED WANTED TO OUTLAW^PRACTICE ARCHERY 






BM 


7 


BM 


7 


N002904 


SOCCER: CALLED FOREIGN-IMMIGRANTS PLAYED IT MOST 






BM 


3 


BM 


g 


N002905 


£»w.v.&n> irrXKU ru bNuUSn ox ROnANS 






BM 


9 


BM 


9 


N002906 


SOCCER :PELE MASTER-FOOLED OPPONENTS BY FAKE MOVES 






BM 


10 


BM 


10 


N003001 


SUPR COURT: CONSTITUTION DESCRIPTION-BRIEF 


BM 


10 


BM 


11 


BM 


11 






P3 


14 


P3 


15 


P3 


15 


N003002 


SUPR COURT: DIFFICULT RESPON FOR CO'JRT MEMBERS 


BM 


11 


BM 


12 


BM 


12 






P3 


15 


P3 


16 


P3 


16 
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XTBH U9MSS ACROSS MSB BIOCXS 



PIBLD SHORT XABBL 

MO3003 5UPR OOUtT: "TRSIR** REFERS TO PROVISIOttS 

11003101 GOODS: DIPT TO MKRKET-ROADS POOR 

11003102 GOODS: YANKEC PCDDLER-TOEAY SALESPERSOfl 
11003104 GOODS: PRIMARY TRAH SCORE (RATER 1) 
N003105 GOODS: PRIMARY TRAIT SCORE (RATER 2) 
11003106 GOODS: PRIMARY TRAIT SCORE 

N003107 GOODS: PRIMARY TRAIT: FIRST RATER ID 

N003108 GOODS: PRIMARY TRAIT: SECOND RATER ID 

11003109 GOODS: PRIMARY TRAIT: RBSOL. RATER ID 

N003201 SIMMER JOB:SOC SECURITY APPUC AT BANK OR POST OFC 

N003202 SUM1ER JOB:BEST TIME TO FIND JOB-BEFORE MID-APRIL 

N003203 SUMMER JOB:NEED S3 CARD TO GET HIRED 

N003204 SUMMER JOB: REFERENCES-PEOPLE WHO KNCW APPLICANT 

N003301 BOBBY :SAYS TALL IS SMART 

N003401 YOUNQ GARDENERS: IN CENTRAL PARK-BEST 

N003501 TOASTER: DRAGON/TOASTER QUALITIES COMPARED 

N003601 MAGIC TRICK: FIRST TIE BLACK THREAD 

N003602 MAGIC TRICK: DIMLY LIT RM, SAY PRODUCE FROM AIR 

N003701 WEB LIFE: THREAD BREAKS-FALLS APART 

N003702 WEB LIFE: MAIN IDEA-PUaTStANIlS NEED EACH OTHER 

N003704 WEB LIFE: PRIMARY TRAIT SCORE (RATER 1) 

W003705 WEB LIFE: PRIMARY TRAIT SCORE (RATER 2) 

N003706 WEB LIFE: PRIMARY TRAIT SCORE 

N003707 WEB LIFE: PRIMARY TRAIT: FIRST RATER ID 

N003708 WEB LIFE: PRIMARY TRAIT: SECOND RATER ID 

N003709 WEB LIFE: PRIMARY TRAIT: RESOL. RATER ID 

N003801 SCOTr:BEST TITLE-SCOTT'S PLAN 

N003802 scon: 6 WEEKS BETWEEN DEPOTS 



AGE 9 

BLOCK ITEM 

BM 12 

P3 16 

BM 14 

BM 15 

BM 16 

BM 16 

BM 16 

BM 16 

BM 16 

BM 16 



N003803 



SCOTT : CACHE-PLACE FOR STORING THINGS 



BN 23 

BN 24 

BN 25 

BN 25 

BN 25 

BN 25 

BN 25 

BN 25 

BO 12 

P4 2 

BO 13 

P4 3 

BO 14 

P4 4 



AGE 
BLOCK 

BM 
P3 

BM 

BM 

BM 

BM 

BM 

BM 

BM 

BM 

BN 
PI 

BN 
PI 

BN 
PI 

BN 
PI 

BN 
P3 



BN 
P2 

BN 
PI 

BN 
PI 

BN 

BN 

BN 

BN 

BN 

BN 

BN 

BN 

BO 
P4 

BO 
P4 

BO 
P4 



13 

ITEM 

13 
17 

14 

15 

16 

16 

16 

16 

16 

16 

12 
7 

13 

8 

14 
9 

15 
10 

16 
19 
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AGE 17 
BLOCK ITEM 



BN 17 



18 
10 

19 
13 

20 
14 

21 

22 

23 

23 

23 

23 

23 

23 

12 
2 

13 
3 

14 
4 



BM 13 

P3 17 

BM 14 

BM 15 

BM 16 

BM 16 

BM 16 

HM 16 

BM 16 

BM 16 

BN 21 

PI 7 

BN 22 

PI 8 

BN 23 

PI 9 

BN 24 

PI 10 

BN 25 

P3 19 



BN 27 

P2 10 

BN 28 

PI 13 

BN 29 

PI 14 

BN 30 

BN 31 

BN 32 

BN 32 

BN 32 

BN 32 

BN 32 

BN 32 

BO 12 

P4 2 

BO 13 

P4 3 

BO 14 

P4 4 
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ZTtM USAOl ACROSS ASCS & BLOCKS 



PIBLD SHORT lABCL 

11003901 SKLFISH PCRSOff :DSSCRimOI9 IN PASSAGE 

N004001 TRIANOLE: FIGURE CfUSMN 

N004002 TRZA1IQLE:NAME FIOUSB AS TRIANGLE 

N004101 NONSENSE WORD l:Kiy&-FlRE 

N004201 MBCMM«0I#:2 MXTIH KXTTSN-FEED 3 OR 4 TKS DAILY 

N004202 fOm^WMlCKT LEAVES FOOD-LEAVE BOW. FOR KIM 

N004301 JAVEUN:KAIN REASON 

N004303 JAVELIN: PRIMARY TRAIT SCORE (RATER 1} 

N004304 JAVELIN: PRIMARY TRAIT SCORE (RATER 1) 

N004305 JAVELIN: PRIMARY TRAIT SCORE 

N004306 JAVELIN: PRIMARY TRAIT: FIRST RATER ID 

N004307 JAVELIN: PRIMARY TRAIT: SECOND RATER ID 

N004308 JAVELIN: PRIMARY TRAIT: RESOL. RATER I.. 

N004401 NAOMI JAMES:HC»f LONG ON SAILING TRIP- 272 DAYS 

N004402 NAOMI J AMES : IMPORTANCE OF TRIP-BROKE VTORLD RECORD 

N004403 NAOMI JANES :V40RST PART OF TRIP- BAD STORM 

N004501 AREA CODES:INFO NY-1 -2 12-5 55-1 21 2 

N004502 AREA CODES: SYRACUSE 1-315-255-6011 

N004601 JOBS 1900: MARTHA THINK-JOB TIRESOME 

N004602 JOBS 1900: JOE FOUND HARD-STAYING IN WX>DS 

N004603 JOBS 1900: JOB AT HOME-ADDIE 

N004605 JOBS: PRIMARY 1*RAIT SCORE (RATER 1} 

N004606 JOBS: PRIMARY TRAIT SCORE (RATER 2} 

N004607 JOBS: PRIMARY TRAIT SCORE 

N004608 JOBS: PRIMARY TRAIT: FIRST RATER ID 

N004609 JOBS: PRIMARY TRAIT: SECOND RATER ID 

N004610 JOBS: PRIMARY TRAIT: RESOL. Rj^rKR ID 

N004701 CARRIER AD: IF INTEREST t MEET REQRMNTS-CALL CIRC 

N004702 CARRIER AD:8 YR OLDS TOO YOUNG FOR JOB 

N004703 CARRIER AD:MUST DELIVER PAPERS BY 7 EACH AM 



AGE 9 
BLOCK ITEM 



BO 
P4 

BO 
P4 

BO 
P4 



17 
16 

18 
12 

19 
13 



BP 
P4 



BP 
P4 



BP 
P4 



7 
9 

8 
10 

9 
11 



HQ 
PI 

BQ 
PI 

BQ 
PI 



10 
15 

11 
16 

12 
17 



AGE 13 

BLOCK ITEM 

BO 16 

P3 14 

BO 15 

BO 15 

BO 17 

P4 22 

BO 18 

P4 18 

BO 19 

P4 19 

BO 20 

BO 21 

BO 21 

BO 21 

BO 21 

BO 21 

BO 21 

BP 7 

P4 15 



BP 8 

P4 16 

BP 9 

P4 17 



BP 10 

BP 11 

BP 12 

BP 13 

BP 14 

BP 15 

BP 15 

BP 15 

BP 15 

BP 15 

BP 15 

BQ 7 

PI 15 

BQ 8 

PI 16 

BQ 9 

PI 17 
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AGE 17 
BLOCK ITEM 



P3 14 



P4 22 

BO 21 

P4 18 

BO 22 

P4 19 

BO 23 

30 24 

BO 24 

BO 24 

BO 24 

BO 24 

BO 24 

P4 15 

P4 16 

P4 17 

BP 20 

BP 21 

BP 22 

BP 23 

BP 24 

BP 25 

BP 25 

BP 25 

BP 25 

BP 25 

BP 25 

PI 15 

PI 16 

PI 17 
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N A E P 



FIELD 


SHORT LABEL 


AGE 
BLOCK 


9 

ITEM 


AGE 
BLOCK 


13 

ITEM 


AGE 

BLOCK 


17 

vm 


11004801 


SZLlOr 3:VIISHE0 HE HAD SOKE HAIR 


BQ 
P3 


13 
19 


BQ 
P3 


10 
20 


P3 


20 


N004901 


COLORADO :aOLD DISCOVER? DOESN'T BELONG 


BQ 
P3 


14 

17 


BQ 
P3 


11 
18 


BQ 
P3 


10 
18 


N005001 


ARTSiBETORE 1940 ARTS WERE ORIENTED TO ELITE 






BQ 
P2 


13 
6 


BQ 
P2 


7 
6 


11005002 


ARTS:PRIVILEQE OP ARISTOCRATIC FEVMREAT WORKS 






BQ 
P2 


14 

7 


BQ 
P2 


8 

7 


N005003 


ARTS:KASS J?ROD NO HARM TO GENUINE ART 






BQ 
P2 


15 
8 


BQ 
P2 


9 
8 


N005101 


Iimn98:WINNIE SHORTER THAN PAMELA-BEST STATEMEriT 


9Q 
P2 


15 
2 


BQ 
P2 


12 
2 


P2 


2 


N005201 


TRArriCiAPPEAR IN COURT TO PLEAD NOT GUILTY 






BQ 
P4 


16 
23 


BQ 


11 


N005202 


TRAPriC:riNE-$3.00 






P4 


24 


BQ 

P4 


1 5 

24 


N005203 


TRAFTICiPAy FINE BY THURS, JUNE 11 






BQ 
P4 


18 


9Q 

ir't 


13 
25 


N005301 


SEALS! GET FOOD ON SHORE-TRDM THEIR FAT 






oQ 


1 Q 






N005302 


SEALS: SURPRISE IN MEXICO-THOUGHT SEALS EXTINCT 








«U 






N005303 


SEALS: MAIN PURPOSE-DESCRIBE SEALS 














N005304 


SEALS: COME SHORE YEARLY-BIRTH TO YOUNG 








22 






N005305 


SEALS: BLUBBER MEANING-FAT 








23 






N005401 


HERO: INTERESTING ARTICLE 






BR 


c 

D 






11005402 


HERO: EASE OF READING ARTICLE 






BR 


c 

V 






N005403 


HERO: MAIN IDEA^SIMON WAS A GREAT HERO 






BR 


7 






NO 054 04 


HERO: FROM WHAT COUNTRY-VENEZUELA 






BR 


g 






N005405 


HERO: TRUE-COLOMBIA ONCE SPANISH 






BR 


9 






N005406 


HERO: MONEY CALLED 'BOLIVARS' 






BR 


10 






N005407 


HERO: GOAL NEVER REACHED-COUMTRIBS JOIN TOGETHER 






BR 


11 






N005501 


BUSINESS: INTERESTING 






BR 


12 


np 

on 


1 5 


N005502 


BUSINESS: EASE OF READING 






BR 


13 


no 


1 1 
1 J 


N005503 


BUSINESS: MAIN PURPOSE- BUSINESS TERMS MEAN 






BR 


14 


BR 




N005504 


BUSINESS: OWE 50 DOLLARS FOR BIKE-A LIABILITY 






BR 


15 


BR 


15 


N005505 


BUSINESS: EXTRA MONEY IS PROFIT 






BR 


16 


BR 


16 


N005601 


TREES: TRAPS ^LLUTANTS-LEAVES 






BR 


17 






N005602 


TREES: CLEANING THE AIR-FILTERING PARTICLES 






BR 


18 






N005603 


TREES: PURPOS GREEN BELT-REDUCE CITY POLLUTION 






BR 


19 






N005701 


GRAPH:MOST PCWER 1980, 1985, 2000-PETROLEUM 






BS 
P3 


19 
28 


BS 
P3 


19 
28 



15^ 



APPENDIX 2 5/23/86 PAGE 9 

ZTEK USA0E ACROSS AOES k BLOCKS N A C P 







AGE 


9 


AGE 


13 


AGE 


17 


PXILD 


SHORT lABEL 


BLOCK 


ITEM 


BLOCK 


ITEM 


BLOCK 


ITEM 


N005702 


aiUaif:IN 2000,im«OF0MSR SUPPLY LESS THAN COAL 






BS 


20 


BS 


20 










P3 


29 


P3 


29 


RU05703 


GRAFHrln ^DDU rcUCIj£AK POWER nOR£ ^ TOTAIj TnAfi 1S71 






BS 


21 


BS 


21 










P3 


30 


P3 


30 


N005801 


E19SUSH DZC:BOOK TELLS WRD MEANINQS-DICTIONARY 


BS 


19 


BS 


22 


BS 


22 






Pi 


18 


Pi 


34 


Pi 


34 


IfD05901 


CaRDCaT:CALL IfUnBO-WKlTE--In d29«1 0B2 






BS 


23 


BS 


23 


N005902 


CARIXATlPZCIURES IRDIC BY "ILLUS** 






BS 


24 


BS 


24 


11006001 


PaOME OIR:STORES SSU. MILK LISTED UNDER DAIRIES 






BS 


25 


BS 


25 




rHUHE DIR:H£rUJKlCK9 lUMXnU On dJRD STy 44J^l!>u2 






BS 


26 


BS 


26 


N006003 


PHONE OIR:STAR TRACKER OPEN TO REPAIR MICROSCOPE 






BS 


27 


BS 


27 


N006101 


WIND SYnaOLS:POR 35 KNOTS-SYMBOL 3 






BS 


30 


BS 


30 










PI 


18 


PI 


18 


n006201 


INDEX:PIND KINS DARIUS INFO ON PG 23 






BS 


31 


BS 


31 










PI 


29 


PI 


29 


N006202 


INDEXrriND CUNBITORM PTONUNCIATION 






BS 


32 


BS 


32 










PI 


30 


PI 


30 


N006203 


INDEX: 1875 FRENCH CONSTZTUHON ZNFO ON FG 233 






BS 


33 


BS 


33 










PI 


31 


PI 


31 


N006204 


INDEX:ALTERNATE HDO./DVTCH EAST ZNDZES-INDONESIA 






BS 


34 


BS 


34 










PI 


32 


PI 


32 


N006205 


ZNDEX:OZSARNAMENT ZN EASl'ERN EUROPE INFO ON PG 279 






uo 


35 


BS 


35 










rl 




rl 




N006301 


CliJTHES SIZES !SnO£ SIZE 8—40—1 






DO 


JO 


nc 


JD 










PI 


11 


Pi 


11 


N006302 


CliOTHES SIZES: 38 SWEATER— 44 






DO 


J / 


nc 


1*7 










PI 


12 


PI 


12 


N006401 


TEXTS: BEST PLACE Tv LOCATE oUliL RUN nSTY— INDEX 










nc 


AO 


N006402 


TEXT3:BEST PLACE FIND I^LTA DEFIN./GEOG-GLOSSARY 






BS 


29 


BS 


29 


N006501 


FIND GUIDE: OPTIONAL BETVTEEN OPPRCSS-ORACLE 


BT 


26 


BT 


26 


BT 


26 






PI 


19 


PI 


19 


PI 


19 


N006601 


TABLE CONTENTS :NOST USEFUL IN AMERICAi: HIST COURSE 






BT 


19 


BT 


19 










PI 


20 


PI 


20 


N006602 


TABLE CONTENTS :AMEiaCAN INDEPENDENCE IN UNIT I 






BT 


20 


BT 


20 










PI 


21 


PI 


21 


N006603 


TABLE CONTENTS :REC0NSTRUCTI0N AFT CIVIL WAR-CHAP. 6 






BT 


21 


BT 


21 










PI 


22 


PI 


22 


K006604 


TABLE CONTENTS: MAJOR TOPIC CHAP.17-HAPPENINGS WWII 






BT 


22 


BT 


22 














Pi 


23 


N006605 


TABLE CONTENTS :MIDDLE EAST MAP, 1958-1970 ON PG,594 






BT 


23 


BT 


23 










PI 


24 


Pi 


24 


N006701 


SCIENCE INDEX :WOLVES FIRST IN BOOK 






BT 


27 


BT 


27 










P4 


26 


P4 


26 


N006801 


MAP:SPANISH IN SOUTH 






BT 


24 


BT 


24 


N006e02 


MAP:PEOPLE SETTLED IN ALASKA-NOT ENOUGH INFORM 






BT 


25 


BT 


25 



ERIC 
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AGE 


9 


AGE 


13 


AGE IV 


riEU) 


SHORT LABEL 


BLOCK 


ITEM 


BLOCK 


ITEM 


BLOCK 


1 XCir 


19006901 


NEHStTV SCKEXXnX-PQ 22 






BT 


36 


BT 


35 


11006902 


EfEKS:lfEArHER FORECAST-PQ 12 






nrr 

oA 


37 


BT 


37 


N006903 


NEWStSTOCX AVERASE&-POS 29-31 






BT 


38 


BT 


38 


N007001 


CATALOG CD:>«Ar INFO GIVES LOCAXlOtMSV 885 C624 






BT 


28 


BT 


28 










PI 


25 


Pi 


25 


11007002 


CAZALOQ CD:PQ FOR CrtHER BOOKS SAKE TOPIC-221 






BT 


29 


BT 


29 










PI 


26 


Pi 


«D 


N007003 


CATALOG CD:AUTaORS OF BOOK-COOPER k SIEDETTTOP 






BT 


30 


BT 


30 










Pi 


27 


Pi 


5*7 


N007004 


CATALOG CDlOfHER HEADim TO Fim> BOOK-SI EDCmOP 






BT 


31 


BT 


31 










PI 


28 


PI 


28 


N007101 


BUS SCHED:LAST BUS IN EVENING LEAVE CITADEL 6:45PH 






BT 


32 


BT 


32 










P3 


22 


P3 


22 


N007102 


BUS SCHED:2ND SAT-AM BUS ARRIVE DOWNItMN 8:15AM 






BT 


33 


BT 


33 










P3 


23 




AO 


N007103 


BUS SCHED:KISS 2:35PM FROM HANCOCK WAIT TILL 3:35 






BT 


34 


BT 


34 










P3 


24 


r J 




N007104 


BUS SCHED:LV RUSTIC WED 9:42AM ARRV EWNTWN 10:15AM 






BT 


35 


BT 


35 










P3 


25 


r J 


ZD 


N007202 


HOLE IN THE BOX: PRIMARY TRAIT SCORE (RATER 1} 


BU 


18 


BU 


18 


BU 


18 






P4 


1 


P4 


1 




1 


N0O7203 


MOLE IN THE BOX: PRIMARY TRAIT SCOPi: (RATER 2) 


.0 


18 


BU 


18 


BU 


18 






P4 


1 


P4 


1 


rH 


1 


N007204 


HOLE IN THE BOX: PRIMARY TRAIT i^CORE 


BU 


18 


BU 


18 


BU 


18 






P4 


1 


P4 


1 


irH 


± 


N007205 


HOLE IN THE BOX: PRIMARY TRAIT: FIRST RATER ID 


BU 


18 


BU 


18 


BU 


18 






P4 


1 


P4 


1 




1 


N007206 


HOLE IN THE BOX: PRIMARY TRAIT: SECOND RATER ID 


BU 


18 


BU 


18 


BU 


18 






P4 


1 


P4 


1 




1 


N007207 


HOLE IN THE BOX: PRIMARY TRAIT: RESOL. RATER ID 


BU 


18 


BU 


18 


BU 


18 






P4 


1 


P4 


1 




1 


N007208 


HOLE IN THE BOX: HOLISTIC SCORE (RATER 1) 


BU 


18 


BU 


18 


BU 


18 






P4 


1 


P4 


1 


VA 




N007209 


HOLE IN THE BOX: HOLISTIC SCORE (RATER 2) 


BU 


18 


BU 


18 


BU 


18 






P4 


1 


P4 


1 


P4 


1 
J 


N007210 


HOLE IN THE BOX: HOLISTIC SCORE 


BU 


18 


BU 


18 


BU 


18 






P4 


1 


P4 


1 


rH 




N007211 


HOLE IN THE BOX: FIRST HOLISTIC RATER ID 


BU 


18 


BU 


18 


BU 


18 






P4 


1 


P4 


1 


irH 


± 


N007212 


HOLE IN THE BOX: SEC>ND HOLISTIC RATER ID 


BU 


18 


BU 


18 


BU 


18 






P4 


1 


P4 


1 


P4 


1 


N007213 


HOLE IN THE BOX: RESOLUTION HOLISTIC RATER ID 


BU 


18 


B'J 


18 


BU 


18 






P4 


1 


P4 


1 


P4 


1 


N007301 


BRIDGERiKIND OF PEOPLE WERE JfTN MEN-FUR TRAPPERS 






BU 


19 


BU 


19 










PI 


1 


PI 


1 


K007302 


BRIDGER:BEST DESCRIBE STORIES-STRETCHED THE TRUHi 






BU 


20 


BU 


20 










PI 


2 


PI 


2 
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PASE 11 



rxKU) 


SHORT LABEL 


AGE 
BLOCK 


9 

ITDl 


AOE 

BLOCK 


13 

ITEM 


AGE 
BLOCK 


17 

ITEM 


NO073O3 


BRIDOBR:SIHIL«-PO«DS OF MUD BOILINO LIKE KUSH 






BU 
PI 


21 
3 


BU 
Pi 


21 
3 


11007304 


BRXtX3ER:1fflO DISCOVERED LAND fKM YELLOHSTONE-COLTCR 






BU 
PI 


22 
4 


BU 
PI 


22 
4 


11007305 


BRIDOERlSHOT MISSED BLIC BECAUSE ELT OUT OF RANGE 






BU 
PI 


23 
5 


BU 
PI 


23 
5 


II007306 


BR1DOER:HYPERBOLE- UlQCS TfiAT HAD NO BOTTOM 






BU 
PI 


24 
6 


BU 
PI 


24 

6 


11007401 


MEMORY: MAW RCASOff HRITB-DeSCRIBE DETAILS OF SUM! 






BU 


25 


BU 


25 


11007402 


MEMOnr: FROWT PORCH IN SUItlER-OOMFORrABLE 






BU 


26 


BU 


26 


11007403 


MEMORY: FAMILY LIFE WOR]>-CLOSE-KNIT 






BU 


27 


BU 


27 


N007404 


MEMORY: SYRUPY MEANINO-HUWIP 






BU 


26 


BU 


28 


N007405 


MEMORY: SET UP SWIN&-SHIPS SAIL ON OCEAN 






BU 


29 


BU 


29 


K007406 


MEMORY: PRIMARY TRAIT SCORE (RATER 1) 






BU 


31 


BU 


31 


N007409 


MEMORY: PRIMARY TRAIT SCORE (RATER 2) 






BU 


31 


BU 


31 


N007410 


MEMORY: PRIMARY TRAIT SCORE 






BU 


31 


BU 


31 


N007411 


MEMORY: SECONDARY TRAIT: CONTENT 






BU 


31 


BU 


31 


NO07412 


MEMORY: SECONDARY TRAIT: FORM 






BU 


31 


BU 


31 


N007413 


MEMORY: SECONDARY TRAIT: SUBJ. REACTION 






BU 


31 


BU 


31 


N007414 


MEMORY: SECONDARY TRAIT: PIECES OF EVIDENCE 






BU 


31 


BU 


31 


N007415 


MEMORY: PRIMARY TRAIT: FIRST RATER ID 






BU 


31 


BU 


31 


NO 0741 6 


MEMORY: PRIMARY TRAIT: SECOND RATER ID 






BU 


31 


BU 


31 


N007417 


MEMORY: PRIMARY TRAIT: RCSOL. RATER ID 






BU 


31 


BU 


?1 


N007501 


TRAVELS :MAN AFRAID- FEARFUL THOUGHTS, NO DANGER 






BV 


29 


BV 


38 


N007504 


TRAVELS: PRIMARY TRAIT SCORE (RATER 1) 






BV 


31 


BV 


40 


N007505 


TRAVELS: PRIMARY TRAIT SCORE (RATER 2) 






BV 


31 


BV 


40 


N007506 


TRAVELS: PRIMARY TRAIT SCORE 






BV 


31 


BV 


40 


N007507 


TRAVELS: SECONDARY TRAIT: CONTENT 






BV 


31 


BV 


40 


N007506 


TRAVELS: SECONDARY TRAIT: FORM 






BV 


31 


BV 


40 


N007509 


TRAVELS: SECONDARY TRAIT: SUBJ, REnCi'ION 






BV 


31 


BV 


40 


NO 07510 


TRAVELS: SECONDARY TRAIT: PIECES OF EVIDENCE 






BV 


31 


BV 


40 


N007511 


TRAVELS: PRIMARY TRAIT: FIRST RATER ID 






BV 


31 


BV 


40 


N007512 


TRAVELS: PRIMARY TRAtT: SECOND RATER 10 






BV 


31 


BV 


40 


NO 0751 3 


TRAVELS: PRIMARY TRAIT: RESOL. RATER ID 






BV 


31 


BV 


40 


N007602 


FLASHLIGHT: PRIMARY TRAIT SCORE (RATER 1) 


BV 


36 


BV 


32 


BV 


41 


N007603 


FLASHLIGHT: PRIMARY TRAIT SCORE (RATER 2) 


BV 


36 


BV 


32 


BV 


41 


N007604 


FLASHIOCGHT: PRIMARY TRAIT SCORE 


BV 


36 


BV 


32 


BV 


41 


N007605 


FLASHLIGHT: PRIMARY TRAIT: FIRST RATER ID 


BV 


36 


BV 


32 


BV 


41 
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AFPIRDIX 2 5/23/86 PAGE 12 

ZTBt USMt ACROSS MttS 6 BUXnCS HASP 

AQE 9 AGE 13 AGE 17 







BLOCK 


ZT'il 


BLOCK 


ITEM 


BLOCK 


ITEM 


11007606 


rXASHUyRT: FRInARx TPAIT: SECOND RATER ZD 


BV 


36 


BV 


32 


BV 


41 


HUU iOV f 


rLASnuZQifT: FRiriAIu 17AIT: RE50L. RATER ZD 


BV 


36 


8V 


32 


BV 


41 


XfUU f fvA 


QHOST STOXXt PRIMARY TRAIT SCORE (RATER 1} 


BW 


39 


BW 


37 


BW 


39 


IfUU / /UJ 


UHOST STORz: HunARY TRAIT SCORE (RATER 2) 


BW 


39 


BW 


37 


BW 


39 


MAATTA4 


GHOST STOSX: PRU'ARY TRAIT SCORE 


BW 


39 


BW 


37 


BW 


39 


1iAAT7Ai( 


GHOST STORT? FRInARx TRAIT: FIRST RATER ZD 


m 


39 


BW 


37 


BW 


39 


MnA77AlC 


vHlDST STuRX: PRInAKx TRAxT: SBOORD RATER ZD 


Bff 


39 


BW 


37 


BW 


39 


fUv f fv f 


GHOST STOSCT: PRIMARY TRAIT: RESOL. RATER ID 


BW 


39 


BW 


37 


BW 


39 


11007801 


oiiaMinAMji.fmz rrorLfi 1 BECOnE SCnClfrARX^iREVf rD 






BW 


38 


BW 


40 


lf007A07 

XVVv /Ov* 


BAS» AAUiAKSK : MUJC rXHO iu3UuJIS PEOFZJf ZZ^DKY CAVES 






BW 


39 


BW 


41 


nuv /Ov J 


DAS AC»XTiAM.K : TRUE-FEOPLE ZZZ LIVF IARGER COMMUN 






BW 


40 


BW 


42 




BASKETMAKERtPCOPZiE ZZZ USED PITHOUSES FOR CEREIONY 






BW 


41 


BW 


43 




FAVORITE MUSZC: PRIMAK7 TRAIT SCORE (RATER 1} 


BW 


43 


BW 


42 


BW 


44 




FAVORITE MUSZC: PRIMARY TRAIT SCORE (RATER 2} 


BW 


43 


BW 


42 


BW 


44 


N007904 


FAVORITE MUSZC: PRIMARY TRAIT SCORE 


BW 


43 


BW 


42 


BW 


44 


11007905 


FAVORITE MUSZC: PRIMARY TRAIT: FZRST RATER ZD 


BW 


43 


BW 


42 


BW 


44 


Mfl07Q0#k 


VXUmDT^V MiraT^* DOTMEW TiCfcT*!**. CK>/VtHT\ «>%i«*«*t« Tn 

riWVIviXC nUolU* rlUflAKz TKAIX* * ithiMSiU RAxER 


BW 


43 


BW 


42 


BW 


44 


11007907 


FAVORITE MUSZC: PRIMARY TRAIT: RESOL. RATER ZD 


BW 


43 


BW 


42 


BW 


44 


11008002 


SPLIT SESSZON: PRIMARY TRAIT SCORE (RATER 1) 






BX 
P2 


16 
9 


BX 
P2 


16 
9 


N0080(3 


SPLIT SESSZON: PRIMARY TRAIT SCORE (RATER 2) 






BX 
P2 


16 
9 


BX 
P2 


16 
9 


mi08004 


SPLIT SESSION: PRIMARY TRAIT SCORE 






BX 
P2 


16 
9 


BX 
P2 


16 
9 


N006005 


SPLIT SESSZON: PRIMARY TTAIT: FZRST RATER ID 






BX 
P2 


16 
9 


BX 
P2 


16 
9 


11008006 


SPLIT SESSZON: PRIMARY TRAIT: SECOND RATER ZD 






BX 
P2 


IC 
9 


BX 
P.? 


16 
9 


11008007 


SPLIT SESSION: PRIMARY TRAIT: RESOL. RATER ZD 






BX 
P2 


16 
9 


BX 
P2 


16 
9 


N008006 


SPLIT SESSZON: HOLISTIC SCORE (RATER 1) 






BX 
P2 


16 
9 


BX 
P2 


16 
9 


19006009 


SPLIT SESSZON: HOLISTZC SCORE (RATER 2) 






BX 
P2 


16 
9 


BX 
P2 


16 
9 


N008010 


SPLIT JESSION: HOLISTZC SCORE 






BX 
P2 


16 
9 


BX 
P2 


16 
9 


N006011 


SPLZT SESSZON: FZRST HOLZSTZC RATER ZD 






BX 
P2 


16 
9 


P2 


16 
9 


N008012 


SPLIT SESSZON: SECOND HOLISTZC RATER ZD 






BX 
P2 


16 
9 


BX 
P2 


16 
9 


N006013 


SPLIT SESSZON: RESOLUTZON HOLZSTZC RATER ZD 






BX 
P2 


16 
9 


BX 
P2 


16 
9 



Er|c I6'3 



APPIRDIX 2 

XTIM USMI ACROSS AOXS k BLOCKS 



5/23/8S 
N A E P 



PAOC 13 



nsLD saoieT label 

MOOHOl CLOSZllQ:PUN-DOORMAlf AT PLAZA HOTEL? NO 



AGE 9 
BLOCK ITCH 



AGE 
BLOCK 



BX 
P4 



13 

ITEM 



17 
7 



AGE 

BLOCK 



BX 

r4 



17 

ITEM 

17 
7 



11008102 CL0SI1I9:PUEI-P0R MORE THAN 50 YEARS? NO 



BX 
P4 



18 
8 



BX 
P4 



18 
8 



N008103 CLOSXllQ:PUN-eND SWII9SINQ CAREER? YES 



BX 
P4 



19 
9 



BX 
P4 



19 
9 



11008104 CLOSZIIQ: PUR-JOB HA'i HELPED HIK7 NO 



BX 
P4 



20 
10 



BX 
P4 



20 
10 



11008105 CLOSIlia:Ptnf-URLOCK SOKE SECRETS? YES 



BX 
P4 



21 
11 



BX 
P4 



21 
11 



11008106 CL0SZ2IS:PUEI-A LOT HINGES Off KI**t^S? YES 



BX 
P4 



22 
12 



BX 
P4 



22 
12 



11008107 CL0SZ1I3:M.MI« PURPOSE-R£FT SWEENEY LEAVES JOB 



BX 



13 



BX 
P4 



23 
13 



N008108 CLOSINOrTONE CAPTION IS CLEVER AND WITTY 



BX 
P4 



24 
14 



BX 



24 
14 



11008201 COtHTAIL: OOALOUSSA WAS KILLED WHILE HUmiNG 



BY 

PS 



BY 
P3 



N008202 COW-TAIL: THEME-PERS(m NOT DEAD TILL FORGOTIEN 



BY 
P3 



BY 
P3 



N00?203 COW-TAIL: OOALOUSSA IS WISE, FAIR FATHER 



BY 
P3 



BY 
P3 



N008204 OOW-TAIL: OGALOUSSA SHAVED HEAD-RETURNED FROM DEAD 



BY 
P2 



BY 
P3 



N008205 0>*-TAIL: PUU GOT SWITCH-ASKED ABT FATHER MISSING 



BY 
P3 



BY 
P3 



10 

5 



N008206 COW-TAIL: IS THIS A GOOD STORY? 



BY 
P3 



BY 
P3 



11 

6 



N008208 COW-TAIL: PRIMARY TRAIT SCORE (RATER 1) 



BY 
P3 



10 

7 



BY 
P3 



12 

7 



N008209 COW-TAIL: PRIMARY TRAIT SCORE (RATE?. 2) 



BY 
P3 



10 

7 



BY 
P3 



12 

1 



N008210 CO^TAIL: PRIMARY TRAIT SCORE 



JY 
P3 



10 

7 



BY 
P3 



12 

7 



N008211 COW-TAIL: SECONDARY TRAIT: COrTTENT 



BY 
P3 



10 

7 



dY 
P3 



12 
7 



N008212 COW-TAIL: SECON^Y TRAIT: FORM 



BY 
P3 



10 

7 



BY 
P3 



12 

7 



N008213 CW-TAIL: SECONDARY TRAIT: SUBJ. RIIACTION 



BY 
P3 



10 

7 



BY 

P.". 



12 

7 



N008214 CW-TAIL: PRIMARY TRAIT: FIRST RATER ID 



BY 
P3 



10 

7 



BY 
P3 



12 

7 



N008215 COW-TAIL: PRIMARY TRAIT: SECOND RATER ID 



BY 
P3 



10 

7 



BY 
P3 



12 

7 



N006216 COW-TAIL: PRIMARY TRAIT: RESOL. RATER ID 



BY 
P3 



10 

7 



BY 
P3 



12 

7 



K008601 CRICKETS: MAKE SOUNDS BY RUBBING WINGS 



OH 
P2 



6 
15 
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APPSNDIX 2 

ITXM USME ACP0S5 AGES k BL0CK3 



PZELO SHORT lABEI. 

11008602 CRICKETS: ItaCH MAKE CHIRPINO SOUNDS-ONL^ MALES 
N008603 CRICKETS: WHERE ARE EARS - IN ntONT LEGS 



AGE 9 
BLOCK ITEM 

BH 7 
P2 16 

BH 8 
P2 17 



N008701 


PICTURE :DOQ UTUIS ON TOP DOGHOUSE-BEST DESCRIPTION 


BH 


9 


K008801 


yraWE'S DOLL:COULai'T FIND-UNDER PORCH 


BJ 


18 


N008901 


DOO:>«Y DOBSlir WMII^JIHINKS DOGS ARE PESTS 


BJ 


21 


11008902 


DOO:CHILD BRUfOIlia HONE SNAKE 


BJ 


22 


11008903 


DOO:IS THIS A GOOD POEM 


BJ 


23 


N008905 


DOG: PRIMARY TRAIT SCORE (RATER 1) 


BJ 


24 


N008906 


DOG: PRIMARY TRAIT SCORE (RATER 2) 


BJ 


24 


N008907 


DOG: PRIMARY TRAIT SCORE 


BJ 


24 


NOoadOi) 


DOG: SECONDARY TRAIT: COOTEOT 


BJ 


24 


N008909 


DOG: SECONDMIY TRAIT: FORM 


BJ 


24 


H008910 


DOG: SECC^nSARY TRAIT: SUBJ. REACTION 


BJ 


24 


MAAflQI 1 

nwvOvxx 


lAA*: FRIHARY TRAIT: FIRST RATER ID 


BJ 


24 


N008912 


DOG: PRIMARY TRAIT: SECOND RATER ID 


BJ 


24 


N008913 


DOG: PRIMARY TRAIT: RESOL. RATER ID 


BJ 


24 


N009001 


FOLKS :WHO ARE THEY-HUMANS WHO LIVE NEARBY 


BK 




11009002 


FOLKS :GRAY FOX THINK-FOLKS WERE SENSIBLE 


BK 


13 


N009003 


FOLKS: MAN VIAS SITTING ON BENCH IN GARDEN 


BK 


14 


N009004 


FOLKS: DO WHEN FOX CAME NEAR-MAN WAS POLITE 


BK 


15 


N009101 


NONSENSE WORD 3 :HABBIES-DOGS 


BK 


16 






P3 


18 


N009201 


PUZZLE 1:bIRD DESCRIBED IN PUZZLE 


BK 


17 






P3 


28 


N009401 


DUAL:VX)RD BAT-2 MEANINGS FOOLED NELL 


BL 


23 


N009601 


xjLrK/xnz 1 •aiin.nu uri STEPS 


BL 


21 






PI 


8 


N009701 


BOXBALL: MASSACHUSETTS TEACHER INVENTED BASKETBALL 


EM 


5 


N009702 


BQXB?*^: PURPOSE OF ARTICLE-HOW aASKETBALL INVENTED 


EM 


6 


N009703 


BOXBALL: TRUE-FOOTBALL INVENTED BEFORE BASKETBALL 


EM 


7 


N009704 


B0XBALL:AT FIRST USED PEACH BASKET FOR GOALS 


EM 


8 


N009705 


BOXBALL: BOTTOMS CUT OUT-TO MAKE IT EASIER 


EM 


9 


N009801 


PUZZLE 3:CHAIR DESCRIBED IN PUZZLE 


BN 


12 


N009901 


DESCRIPTION 3: PERSON HAS SEEN TOY MM/Y TIMES 


BN 


IJ 


NOlOOOl 


DOG k SHADOW:UKED READING IT 


BN 


17 






Pi 


5 



AGE 13 
BLOCK ITEM 



5/23/86 PACK 14 
N A E P 

AGE 17 
BLOCK ITEM 
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APPTOIX 2 

ITEM USMS ACROSS AGES BLOCKS 



FIELD SHORT LABEL 

11010002 DOQ fc SHADGfH: SAK HIHSCI^ IN THE STREAM 

N010003 XXy^ SRAZXW: TEACHES LESSC2f-<SlEED DOESN'T PAY 

NOIOIOI SAm3l>nCH:LIKE0 READINQ IT 

N010102 SANI»nCH:NAMED AFTER PERSON WK> INVEirrED IT 

N010103 SANDIfICH:l«ANTED MEAT IN BREAD TO EAT AND GAMBLE 

N010201 DESCRIPTION llOXm DESCRIBEL IN PASSAGE 

N010301 SN0HKAN:BE5T DESCRIPTION-SONEONE MADE SNOWMAN 

KOI 0401 TOOTH TROUBLE: SPEAKER-CHILD 

N010402 TOOTH TROUBLE: TRUE-PULLED LOOSE TOOTH 

N010403 TOOTH TROUBLE: "WOT ME, WMTT PRDCE IT" SAME-NO GRO 

S010501 QUICXSA!n>:HO^? TEST FOR IT-PCKS WTTK A STICK 

N010502 QUICXSAND:MAIN PURPOSE-TO TELL WAYS AVOID DANGER 

N010503 QUICKSAND:IT ir SOUPY SAND YOU CAN'T STAND ON 

N010504 QUICKSAND: IF STEP IN, LIE ON BACK k STRETCH OUT ARM 

N01Q601 TKAD:CANDIDATES for FRES NOT ALLOWED GIVE GIFTS 

N010602 THAD:MMG6IE THOOSHT THAD GOOD BUT NEED HER HELP 

N010603 THAD:MASSIVE ST/^EDE-LOT OF PEOPLE RUSHING 

N010604 THAD: EXAGGERATED-CAN DO EVERYTHING IN YELLOW «AGES 

N010605 THAD:MAGGIE FIRST HELPED THAD WITH SPEECH 

N010701 SENTENCE 3:N0ST SENSE-BALL ROLLED .COWN THE STREET 

N010801 ANGRY: CHILD COMES OUT WHEN FEELS BETTER 

N010901 STARS UNSEEN: LIKED READING IT 

N0X0902 STARS UNSEEN:STAR BECOMES DEAD BY USING UP FUEL 

N010903 STARS UNSEEN:MAIN IDEAr-STARS EXIST-WE CAN'T SEE 

N010904 STARS UNSEEN: GRAVITY OF DEAD STARS-PUSH PULL 

tlOllOOl REPORTER: WHO WAS ERNIE PYLE-NEWSPAPER REPORTER 

N011002 REPORTER :HCW PYLES WRITING CHANGE-TROOP MVMNT&GNRL 

N011003 REPORTER: HAPPENED TO PYLE-FAMOUS Rf.PORTER 

N011004 REPORTER: WHY PYLE CHANGE NEWS-HEMEMBED DEATH SOLDI 



AGE 9 

BLOCK ITEM 

BN 16 

PI 6 

BN 19 

p: 7 

BN 20 

PI 9 

BN 21 

PI 10 

BN 22 

PI 11 

BO 16 

P3 20 

BO 15 

P2 9 

BO ZZ 

BO 21 

BO 22 

BP 10 

P2 3 

BP 11 

P2 4 

BP 12 

P2 5 

BP 13 

P2 6 

BP 14 

BP 15 

BP 16 

BP 17 

BP 18 

BP 19 

BQ 16 

P3 29 

BQ 17 

PQ 18 

BQ 19 

BQ 20 



AGE 13 
BLOCK ITEM 



5/23/86 PAGE 15 
N A E P 

AGE 17 
BLOCK ITEM 



BR 
BR 
BR 
BR 
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APPfiCDIX 2 

ITEN USAGE ACROSS AGES & BLtXHCS 

PZILD SHQRT lABEL 

sioiiioi kucd op bx:A!xh>spkerc from science DCOK 

N011201 ]»QS'QUAL:BrrrEN BY DOG, DISAGREE 

R011301 SKUraC CABBA3E:NAH'r-SKELLS LIKE SKUNK,XX)OXS CABBAGE 

N011302 SKUNK CAUUAGE:HARD TO SEE-^DDEN UNDER HOOD 

11011401 BREATHIM3:TRU£~BL0OD MOVES OXYGEN 

11011402 BREAIHIlia: HGW AIR MOVES TO LUI9QS-THR0UGH WINDPIPE 
N011403 BREATHIIf8:FUllCTI0N OP AIR SACS IN LUIIGS-02 FROM LU 
N011404 BR£AniIlVS:CD2 FORK IN BODY-<ELLS PORK C02 WASTE 
N011501 DICTIONARY :TO FIND WORD MEANIWCj-DICYlONARY BEST 
N011601 DICTi:»VaiY:i:«PINITION TQME>A lARGE BOOK 

N011602 DICTIONARY :TOH0RROW SYLLABI CATED-TO NOR ROW 

N011603 DICTIONARY: PLURAL IS TONSILLECTOMIES 

N011604 DICTIONARY:TOLERANCE IS A NOUN 

N011605 DICTIONARY:TOtTIC-MAKES YOU FEEL BETTER 

N011701 WHICH WORD COMES PIRST IN DICTIONARY- FLEA 

N011801 ENCYCLOPEDIAS 2:WASHINGT0N IN VOL 11 

N011901 INDEXrPIND OUT ABOUT SAIWON-PGS 84 t85 

N011902 INDEX:ALT£RNAT£ INFO;RAIIiU)S-TRAVEL k TRANSPORT 

N011S03 INDEX: FIND MAP OP SNAKE RIVER-PG 84 

N011904 INDEXrPIND MAP S. AMERICAN RAIN FORESTS-PG. 119 

NO7.2001 DEQARATION OP INDEPENDENCE: BEST INFO ENCYCLOPEDI 

N012101 a)t«:WHAT DOES HPPE ACTUALLY SPELL-GOOD 

N012201 DICTIONARY: ?LUME IS FEATHER 

N012202 DICTIOtlARY:M0RE THAN 1 PLOWMAN IS PLOWMEN 

N012203 DICTIONARY: PLUNDER-ROB 

N012204 DICTIONARY: PLUM-IMPORTANT WORK 



5/23/86 PAGE 16 
N A E P 

AGE 9 A3E 13 AGE 17 

BLOCK ITEM BLOCK ITEM BLOCK ITEM 

BR 9 

PI 36 

BR 10 

P2 13 

BR 11 

P3 21 

BR 12 

P3 22 

BR 13 

BR 14 

BR 15 

BR 16 

BT 27 

BS 21 

P3 23 

BS 22 

P3 24 

BS 23 

P3 25 

BS 24 

P3 26 

BS 25 

P3 27 

BS 30 

P4 18 

BS 31 

BS 26 

PI 24 

BS 27 

PI 25 

BS 28 

PI 26 

BS 29 

PI 27 

BS 32 

BS 33 

BT 19 

PI 28 

BT 20 

PI 29 

BT 21 

PI 30 

BT 22 

PI 31 



ERIC 
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APPBCDZX 2 

ITEN USMC ACROSS AO£S fc BLOCKS 



5/23/86 PAGE 17 
N A E P 



TlttD 


SHOSer lABEL 


AGE 
BLOCK 


9 

ITEM 


NO12301 


MUSHROOM: 3 PARTS-CAP, STEM, GILLS 


BS 
PI 


20 
12 


NOX2401 


IMDEXlALPHA LIST OF TOPICS AND PAGE NUMBERS 


BT 


24 


N012501 


HHALE food: IN/O FOUND IN ENCYCLOPEDIA 


BT 


25 


N012601 


ROTOP.rBBST PIACE J-'INO INFC-DICTIONARY ROTOR 


BT 
PI 


23 
13 


N012701 


ENCYCLOPEDIA: INFO CX7 MEXICO IN VOLUME 6 


BT 
PI 


28 
32 


N012702 


ENCYLOPEDIA:INFO ON INVENTIONS OF EDISCW IN VOL.3 


BT 
PI 


29 
33 


{9012703 


ENCYL0PEDIA:INFO ON IOWA FARM PRODUCTS IN VOL. 5 


BT 
PI 


30 
34 


{9012704 


ENCYCLOPEDIA: INFO ON N.Y. RIVERS ^ LAKES IN VOL. 7 


BT 
PI 


31 
35 


11012801 


GRAPH: SPENT MOST ON A BOOK 


BT 
PI 


32 
20 


{9012802 


G2iAPH:R£C0RD COST $2.50 


BT 
PI 


33 
21 


19012803 


GRAPH: 5 ITEMS COST MORE THAN PAINTBRUSH 


BT 
PI 


34 
22 


19012804 


GRAPH: SPENT SAME AMOUNT ON PAINTS, BIKE PARTS 


BT 
PI 


35 
23 


19012901 


TIMOTHY: 3 TEENAGERS TALKING ABOUT HEAT 


BU 


19 


190130G1 


OIL SPILL: WHAT IS THE GEORGIA-A SHIP 


BU 


20 


N013002 


OIL SPILL: WHERE WAS SPILL-5 MILES FROM BEACH 


BU 


21 


N013003 


OIL SPILL:WHY LOGS NO STOP OIIr-HIGH WAVES 


BU 


22 


19013004 


OIL SPILL: WHAT ARE PEOPLE ASKED TO DO-CLEAN BEACH 


BU 


23 


N013101 


THE COLD: BOY LEFT SHADOW-FROZE TO SIDE OF HOUSE 


BU 
PI 


24 
1 


N013102 


THE COLD:GIRLS FIGHT WITH MELTED WORDS 


BU 
PI 


25 
2 


N013103 


YHE COLD:DUCKS FLY AWAY WITH POND 


BU 
PI 


26 

3 


N013104 


THE C0LD:WR1TER MAKES STORY SOUND PLAYFUL ( FUNNY 


BU 
PI 


27 
4 


N013201 


BULLFIGHT: BULL CHARGES CAPE MOTION 


BV 
P4 


29 
1. 


N013301 


DESCRIPTION 2: UNHAPPY PERSON DESCRIBED IN PASSAGE 


BV 
PI 


30 
14 


N013401 


FROM THE PLANET :B0TCHIK FELT ANNOYED AND UPSET 


BV 


31 


N013402 


FROM THE PLANET :TH0UGHT NO LIFE-THICK CLOUD COVER 


BV 


32 


N013403 


FROM THE PLANET: IN GLASS CAGE WAS A HUMAN BEING 


BV 


33 


N013501 


CRIME:HARD TO PROVE OWN BIKE IF REPAINIED & # GONE 


BW 
P4 


37 
7 



AGE 13 
BLOCK ITEM 



AGE 17 
BLOCK ITEM 
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APPBROIX 2 

ITEM USjySE ACROSS MBS k BtX>CKS 



PIBLD SHORT LABEL 

W013502 CRIME:MMII PURPOSED GIVE SECRET WAY TO MARK BIKE 

N013601 Sia299I]9S/STAR:PEOPI£ LIKE PIG IP LAZY AND RUDE 

N013602 SNIIVSINS/STAR: PEOPLE SHOUZiD DlfPER-^TRY BE BETTER 

M013603 SWniGIllQ/STAR: LIKE 4 DOESN'T RHYME WITH OTHERS 

R013701 OLD MAN:STORY TELLS BOW MAN LOOKS 

N013901 SAVma ENERGY: MAIN IDGA-CORSERVE OIL ( NAT GAS 

N013902 SAVmS ENERGY: SOURCE MOST ENERGY-OIL ( NAT OAS 

N013903 SAVING ENERGY: HRAT CAN SOLAR ENERGY PROVIDB-HEAT 

N014001 IK3NSENSE l«>RD 3:TUV^FAPER 

N014101 SENTENCE l:nOST SENSE-BLEVf HOUSE DOWN 

N014201 TIMOTHY 2:TBENAGBRS STANDING IN CIRCLES 

N014301 FRONTIER VX3MEN: BEST DESCRIBES WOHEN-440RKED HARD 

N014302 FRONTIER WOMEN: ACTIVITIES PERFORMED-MAKE TOOLSfcPL 

N014303 FRONTIER WOMEN: MADE FROM ANHL HORNS/BONES-TOOLS 

N014501 CONNECT DOTSrALONS LINE, CONNECT DOTS 

N014502 CONNECT DOTS: DRAW LINE TO TOUCH CIRCLES 

N014503 CONNECT DOTSrWRITE 3 IN EACH CIRCLE 

N014702 PLANTS: PRIMARY TRAIT SCORE (RATER 1) 

N014703 PLANTS: PRIMARY TRAIT SCORE (RATER 2) 

N014704 PLANTS: PRIMARY TRAIT SCORE 

N014705 PLANTS: PRIMARY TRAIT: FIRST RATER ID 

N014706 PLANTS: PRIMARY TRAIT: SECOND RATER ID 

N014707 PLANTS: PRIMARY TRAIT: RESOL. RATER ID 

N014802 SPACESHIP: PRIMARY TRAIT SCORE (RATER 1) 

N014803 SPACEi^HlP: PRIMARY TRAIT SCORE (RATER 2) 

N014804 SPACESHIP: PRIMARY TRAIT SCORE 

N014805 SPACESHIP: PRIMARY TRAIT: FIRST RATER ID 

N014806 SPACESHIP: PRIMARY TRAIT: SECOND RATER ID 

NU14807 SPACESHIP: PRIMARY TRAIT: RESOL. RATER ID 

N014902 AUNT MAY; PRIMARY TRAIT SCORE (RATER 1) 

N014903 AUNT MAY: PRIMARY TRAIT SCORE (RATER 2) 



N014904 



AUNT MAY: PRIMARY TRAIT SCORE 



N014905 AUNT MAY: SECONDARY TRAIT: PIECES OF EVIDENCE 



AGE 9 

BLOCK ITEM 

BW 38 
P4 8 

BW 40 

BW 41 

BW 42 

BX 17 

BX 18 

BX 19 

BX 20 

BM 13 

P2 14 

BQ 21 

BV 34 

BN 14 

BN 15 

BN 16 

BV 35 

BV 35 

BV 35 

BC 23 

3C 23 

BC 23 

BC 23 

BC 23 

BC 23 

BE 10 

BE 10 

BE 10 

BE IC 

BE 10 

BE 10 

BX 16 
P2 a 

BX 16 
P2 8 

BX 16 
P2 8 

BX 16 
P2 8 



AGE 13 
BLOCK ITEM 



5/23/86 PAGE 18 
N A E P 

AGE 17 
BLOCK ITEM 



ERLC 



16^ 



APPSfDXX 2 

XTKM USMB ACBOSS AQSS k BLOCKS 



PISLD SHORT LABEL 

N014906 Km MAY: PRXMARy TRAZT: FIRST RATER ID 

N014907 AXnrr may: primary TRAIT: SECCKID RATER ID 

N014908 KJm MAY: PRIMARY TRAIT: RBSOL. RATER ID 

N014909 AUNT MAY: HOLISTIC SCORE (RATER 1) 

N014910 AtnVT MAY: HOLISTIC SCORE (RATER 2) 

N014911 AUHr MAY: HOLISTIC SCORE 

M014912 AUNT MAY: FIRST HOLISTIC IwATER ID 

N014913 AUNT MAY: SECOND HOLISTXC RATER ID 

N014914 ' .iT MAY: KLSOUmC^! HOLISTIC RATER ID 

ZiOlSlCl BLACK ELK: THINK nASICHuS nEHE GHEEux 

N015102 BLACK ELK: ViHO WERE THE WASICHUS 

N015103 BLACK ELK:ORINKS WATERS DREAM PREDICT 

11015104 BLACK ELK: MAIN PURPOSE OF STORY 

N015201 PEOPLE LEARN TO READ: IN SCHOOL 

11015501 CHAMONIX: LIKED READING IT 

N015502 CHAICmx: WHY SO LONG TO REACH*4nNDS TOO STRONG 

N015503 CHAMONIX: DEVOUASSOU-MAN WHO FOUND CLIMBERS 

N015504 CHAMONIX: DESMAISON SURVIVE BY MENlAIy^HYS STRNGTH 

N015S05 CHAMONIX: WHY DESMAISOK CRY-OVERCOME SUFFERING, JOY 

N015901 HIGH TECH PIZZA: WHY PIZZA 

N015902 HIGH TECH PIZZA: INTERMEDIATE STAGE 

N015903 HxGH VECH PIZZA: CORNSTARCH USED 

N015905 HIGH TECH PIZZA: PRIMARY TRAIT SCORE (RATER 1) 

N015906 HIGH TECH PIZZA: PRIMARY TRAIT SCORE (RATER 2) 

N015907 HIGH TECH PIZZA: PRIMARY TRAIT SCORE 

N015908 HIGH TECH PIZZA: PRIMARY TRAIT: FIRST RATER ID 

N015909 HIGH TECH PIZZA: PRIMARY TRAIT: SECOND RATER ID 

N015910 HIGH TECH PIZZA: PRIMARY TRAIT: RESOL. RATER ID 

N016001 VOTING: MAIN PURPOSE 

N01600' VOTING: MEANING OF SI 'FRAGE 

N016003 VOTING: FIRST CONGRESSWDMEN 



AGE 9 

BLOCK ITEM 

6X 16 

P2 8 

BX 16 

P2 8 

BX 16 

P2 8 

BX 16 

P2 8 

BX 16 

P2 8 

BX 16 

P2 8 

BX 16 

P2 8 

BX 16 

P2 8 

BX 16 

P2 8 



AGE 13 
BLOCK ITEM 



5/23/86 PAGE 19 
N A E P 

AGE 17 
BLOCK ITEM 



on X I 

BR 18 

BR 19 

BR 20 

BN 26 

BP 15 

BP 16 

BP 17 

BP 18 

BP 19 

BQ 14 

BQ 15 

BQ 16 

BQ 17 

BQ 17 

BQ 17 

BQ 17 

BQ 17 

BQ 17 

BO 15 

BO 16 

BO 17 



ERLC 
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APPENDIX 2 

ITEM USA3C ACROSS AGES k BLOCKS 



PI8U) SHORT lABSL 

II016004 VDTHW: DISASTER AT TRIANOLB SHIRTWAIST 

N016005 VOnnS: ROSE SCHREIDGRHMI SAY 

N016006 VOniW: m I HEI-PED SUTFRASIST CAUSE 

N017001 THE CHIP: MAIN IDEA 

NO170O2 THE CHIP: WIDESPREAD RESULT 

N017003 THE CHIP: MEANI19S OF TRIFLING 

N018002 SPACE PROGRAM: PRIMARY TRAIT SCORE (RATER 1) 

IIO18C03 SPACE PROORAK: PRIMMW TRAIT JCORE (RATER 2) 

N018004 SPACE PROGRAM: PRIMARY TRAIT SCORE 

N018005 SPACE PROGRAM: PRIMARY TRAIT: FIRST RATER ID 

rwl8006 SPACE PROGRAM: PRIMARY TRAIT: SECOND RATER ID 

N018007 SPACE PROGRAM: PRIMARY TRAIT: RESOL. RATER ID 

N019002 JOS APPLICATION: PRIMARY TRAIT SCORE (RATER 1) 

N019003 JOB APPLICATION: PRIMARY TRAIT SCORE (RATER 2) 

NO19004 JOB APPLICATION: PRIMARY TRAIT SCORE 

N019005 JOB APPLICATION: SECONDAi^Y TRAIT: PERSONAL INFO 

N019006 JOB APPLICATION: SECONDARY TRAIT: JOB INFO 

N019007 JOB APPLICATION: PRIMARY TRAIT: FIRST RATER ID 

N019008 JOB APPLICATION: PRIMARY TRAIT: SECOND RATER ID 

N019009 JOB APPLICATION: PRIMARY TRAIT: RESOL. RATER ID 

N020002 LETTER TO UNCLE: PRIMARY TRAIT SCORE (RATER 1) 

N020003 LETTER TO UNCLE: PRIMARY TRAIT SCORE (RATER 2) 

N020004 LETTER TO UNCLE: PRIMARY TRAIT SCORE 

N020005 LETTER TO UNCLE: SECONDARY TRAIT: MADE NOTES 

N020006 LETTER TO UNCLE: PRIMARY TRAIT: FIRST RATER ID 

N020007 LETTER TO UNCLE: PRIMARY TRAIT: SECOND RATER ID 

N020008 LETTER TO UNCLE: PRIMARY TRAIT: RESOL. RATER ID 

N021002 B^vE LANE: PRIMARY TRAIT SCORE (RATER 1) 

N021003 BIKE LANE: PRIMARY TRAIT SCORE (RATER 2) 

NO 21004 BIKE LANE: PRIMARY TRAIT SCORE 

N021005 BIKE LANE: PRIMARY TRAIT: FIRST RATER ID 

N021006 BIKE LANE: PRIMARY TRAIT: SECOND .lATER ID 

N021007 BIKE LANE: PRIMARY TRAIT: RESOL. RATER ID 



5/23/36 PAGE 
N A E P 



AGE 9 
BLOCK ITEM 



AGE 13 
BLOCK ITEM 



AGE 


17 


BLOCK 


ITEM 


BO 


18 


BO 


19 


BO 


20 


BH 


7 


6H 


8 


BH 


9 


BE 


10 


BE 


10 


BE 


10 


BE 


10 


BE 


10 


BE 


10 


BE 


11 


BE 


11 


BE 


11 


BE 


11 


BE 


11 


BE 


11 


BE 


11 


9E 


11 


BF 


7 


BF 


7 


BF 


7 


BF 


7 


BF 


7 


BF 


7 


BF 


7 


BG 


7 


BG 


7 


BG 


7 


BG 


7 


BG 


7 


BG 


7 
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Apmozx 2 

ZTBK USA9E ACROSS AGES k BLOCKS 



rZBLD SHORT LABEL 

SOOOlOl TIME SPCTT IN ENOLISH CLASS LEARNING TO WRITE 

S000201 REPORTS AND PAPERS VmiTTEN FOR SCHOOL LAST 6 WEEKS 

S000301 WRITINGS DONE LAST WEEK FOR SOCIAL STUDIES CLASS 



AGE 9 
BLOCK ITEM 

BA 1 

BU 8 



BA 

BU 



AGE 
BLOCK 



BA 

BU 



BA 3 
BU 10 



BA 

BU 



BA 
BU 



13 

ITEM 



3 
10 



5/23/86 
N A E P 

AGE 
BLOCK 

BA 

BU 

BA 
BU 

BA 

BU 



PAGE 21 



17 

ITEM 

1 
8 

2 
9 

3 
10 



S000401 WRITINaS DONE LAST WEEK FOR SCIENCE 



BA 4 
BU 11 



BA 

BU 



4 
11 



BA 

BU 



4 

11 



S000501 WRITIN3S DONE LAST WEEK II0N->SCHOOL RELATED 
S000601 WHEN WRITING HOW OFTEN TEAOIER ASK* TO MAKE NOTES 



BA 



BA 
BU 



BA 



BA 

BU 



BA 



BA 

BU 



S000602 WHEN WRITING HOW OFTEN TEACHER ASKS MAKE OUTLINE 



BA 
BU 



BA 

BU 



BA 

BU 



S000603 \ HEN WRITING HCW OFTEN TEACHER ASKS NOTE CHANGES 



BA 
BU 



BA 

BU 



BA 
BU 



S000604 WHEN WRITING HCW OFTEN TEACHER ASKS TALK TEACHER 



S000605 WHEN WRITING HOW OFTEN TEACHER ASKS TALK MATES 



S000606 WHEN WRITING HOW OFTEN TEACHER ASK REDO BEFOR GHD 



BA 
BU 



BA 10 

oU 5 

BA 11 

BU 6 



BA 
BU 



BA 

OU 



BA 
BU 



10 



11 
6 



BA 
BU 



BA 

OU 



BA 
BU 



9 

4 

10 



11 
6 



S000607 WHEN WRITING HOW OFTEN TEACHER ASK REDO AFTER GRD 

5000701 HOW OFTEN IS TRUE: WRITING IS IMPORTANT 

5000702 HOW OFTEN IS TRUE: WRITING HELPS LEARN ABOUT SELF 

5000703 HOW OFTEN IS TRUE: WRITING REMINDS ABOUT THINGS 
3000704 HOW OFTEN IS TRUE: WRITING HELPS ME STUDY 
5000705 HOW OFTEN IS TRUE: WRITING HELPS NEW IDEAS 
S000901 WHEN WRITING HCW OFTEN ASK SELF SUBJECT PAPER 



BA 12 
BU 7 

1 

2 

3 

4 

5 

6 
1 



BB 

BB 

BB 

BB 

BB 

BB 
BV 



BA 

BU 

BB 

BB 

BB 

BB 

BB 

BB 
BV 



12 
7 

1 

2 

3 

4 

5 

6 
1 



BA 
BU 

BB 

BB 

BB 

BB 

BB 

BB 
BV 



12 
7 

1 

2 

3 

4 

5 

6 
1 



S000902 WHEN WRITING HCW OFTEN LOOK UP FACTS IN BOOKS 



BB 
BV 



BB 
BV 



BB 
BV 



S000903 WHEN WRITING HOW OFTEN THINK BEFORE WRITING 



BB 
BV 



BB 
BV 



BB 
BV 



S000904 WHEN WRITING HOW OFTEN THINK ABOUT ORGANIZATION 



BB 
BV 



BB 
BV 



BB 
BV 



S000905 WHEN WRITING HOW OFTEN USE DIFF STYLES PER PERSON 



BB 10 

B\^ 5 



BB 
BV 



10 

5 



BB 
BV 



10 

5 



S000906 WHEN WRIT:NG HOW OFTEN MAKE CHANGES AS YOU WRITE 



BB 11 
BV 6 



BB 
BV 



11 
6 



BB 

BV 



11 
6 



S000907 WHEN WRITING HOW OFTEN MAKE CHANGES AFTER WRITING 



BB 12 
BV 7 



BB 
BV 



12 
7 



BB 
BV 



12 
7 



SOOlOOl HOW OFTEN HAVE YOU SHOWN FRIENDS YOUR WRITINGS 



ERLC 



S001002 



HOW OFTEN HAVE PAPERS BEEN PRINTED IN SCHOOL PAPER 
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BB 13 

BQ 7 

BB 14 

BQ 8 



BB 
BY 



BB 
BY 



13 
1 

14 

2 



BB 
BY 



BB 
BY 



13 
1 

14 
2 



APPDTOIX 1 5/23/86 PAOE 22 

ITCH USAGE ACROSS AGES 6 BLOCKS N A E P 



FZSLD 


SHORT lABEL 


AGE 
BLOCK 


9 

ITEM 


AGE 
BLOCK 


13 

ITEM 


AGE 17 
BLOCK ITE^ 


S001003 


HGW OFTEN DOES YOUR FAMILY READ YOUR PAPERS 


BB 
BQ 


15 
9 


BB 
BY 


15 

3 


BB 
BY 


15 

3 


SOOllOl 


HCW OFTEN DOES FAMILY LIST THINtSS TO BUY OR DO 


BC 


1 


BC 


1 


BC 


1 


S001102 


Ka?> OFTEN DOES FAMILY COPY RECIPES OR DIRECTIONS 


BC 


2 


BC 


2 


BC 


2 


S001103 


HOW oriiai DOES family fill OUT ORDER BLANKS 


BC 


3 


BC 


3 


BC 


3 


S001104 


HCW OFTEN DOES FAMILY WRITE CHECKS/KEEP BUDGETS 


BC 


4 


BC 


4 


BC 


4 


S001105 


HOW OFTEN DOES FAMILY KEEP DIARIES OR JOURNALS 


BC 


5 


BC 


5 


3C 


5 


S001106 


HOIf OFTEN DOES FAMILY WORK CROSSWORD PUZZLE 


BC 


6 


BC 


6 


BC 


b 


S001201 


HOW OFTEN IS TRUE: I LIKE TO WRITE 


BC 


7 


BC 


7 


BC 


7 


S001202 


HCW OFTEN IS TRUE: I AM A GOOD WRITER 


BC 


8 


BC 


8 


BC 


8 


S001203 


HOW OFTEN IS TRUE: THINK WRITING IS WASTE OF TIME 


BC 


9 


BC 


9 


BC 


9 


S001204 


KJW OFTEN IS TRUE: PEOPLE LIKE WHAT I WRITE 


BC 


10 


BC 


10 


BC 


10 


S001205 


HOW OFTEN IS TRUE: WRITE ON OWN AWAY FROM SCHOOL 


BC 


11 


BC 


11 


BC 


11 


SOO1206 


HOW OFTEN IS TRUE: DISLIKE WRITING TO BE GRADED 


BC 


12 


BC 


12 


BC 


12 


S001207 


KOW OFTEN IS TRUE: WOULDOT WRITE IF NOT FOR SCHOOL 


BC 


13 


BC 


13 


BC 


13 


S001301 


HOW OFTEN IS TRUE: MOVE SEWTENCES AROUND 


BC 
BX 


14 
1 


BC 
BX 


14 
1 


BC 
BX 


14 

1 


S001302 


HOW OFTEN IS TRUE: ADD NEW IDEAS OR INFORMATION 


BC 
BX 


15 
2 


BC 
BX 


15 
2 


BC 
BX 


15 
2 


S001303 


HOW OFTEN IS TRUE: TAKE OUT UNDESIRED PARTS 


BC 
BX 


16 
3 


BC 
BX 


16 

3 


2K. 
BX 


10 

3 


S001304 


HOW OFTEN IS TRUE: CHANGE WORDS 


BC 
BX 


17 
4 


BC 
BX 


17 
4 


BC 
BX 


1 / 

4 


S001305 


HOW OFTEN IS TRUE: CORi^C^ SPELLING MISTAKES 


BC 
BX 


18 
5 


BC 
BX 


18 
5 


BX 


i 0 


S001306 


HOW OFTEN IS TRUE: CORRECT GRAMMAR MISTAKES 


BC 
BX 


19 

6 


BC 
BX 


19 
6 


BX 


1 Q 

l7 

6 


S001307 


HOW OFTEN IS TRUE: CORRECT PUNCTUATION MISTAKES 


BC 
BX 


20 

7 


BC 
BX 


20 

7 


BC 
BX 


20 

7 


5001308 


HOW OFTEN IS TRUE: REWRITE MOST OF PAPER 


BC 
BX 


2? 
8 


BC 
BX 


21 

8 


BC 
BX 


21 

8 


S001309 


HOW OFTEN IS TRUE: THROW OUT AND START OVER 


BC 
BX 


22 
9 


BC 
BX 


22 

9 


BC 
BX 


22 

9 


S001401 


HCW OFTEN IS TRUE: GOOD WRITING GETS A BETTER JOB 


BD 


1 


BD 


1 


BD 


1 


S001402 


HOW OFTEN IS TRUE: GOOD WRITING MORE INFLUENTIAL 


BD 


2 


BD 


2 


BD 


2 


S001501 


HOW OFTEN TRUE: WRITING HELPS THINK MORE CLEARLY 


BD 


3 


BD 


3 


BD 


3 


S001502 


HOW OFTEN TRUE: WRITING HELPS TELL OTHERS THINKING 


BD 


4 


OU 


4 


3D 


4 


S001503 


HOW OFTEN TKiE: WRITII'o HELPS TELL OTHERS PbELirXSS 


BD 


5 


BD 


5 


BD 


c 


S001504 


HOW OFTEN TRUE: WRITING HELPS UNDERSTAND MYSELF 


BD 


6 


BD 


6 


BD 


6 


S001601 


HOW OFTEN DO YOU LIST THINGS TO BUY 


BD 


7 


BD 


7 


BD 


7 


S001602 


HOW OFTEN DO YOU COPY RECIPES OR DIRECTIONS 


BD 


8 


BD 


8 


BD 


8 



Er|c ^"^3 
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ACROSS AGES fc BLOCKS 
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N A E P 



FIELD 


SHORT lABKL 


AGE 
BLOCK 


9 

ITEM 


AGE 

BLOCK 


13 

ITEM 


AGE 

BLOCK 


17 
ITEf 


S001603 


HCT' OFTEN DO tOU PILL OUT ORDER BIANKS 


BD 


9 


BD 


9 


BD 


9 


S001604 


HOH OPTEN DO YOU KEEP A DIARY OR JOURNAL 


BD 


10 


BD 


10 


BD 


10 


S001605 


HCW OFTEN DO YOU DO A CROSSWORD PUZZLE 


BD 


11 


BD 


11 


BD 


11 


S001606 


HOW OFTEN DO YOU HELP OTHER STUDENTS WITH WRITING 


BD 


12 


BD 


12 


BD 


12 


S001607 


HCW OFTEN DO YOU WRITE ABOUT WHAT YOU HAVE READ 


BD 


13 


BD 


13 


ou 




SOOloOB 


HOW OFTEN DO YOU WRITE PAPERS TOO PERS(M2AL TO SHOW 


BD 


14 


BD 


14 


BD 


14 


S001609 


HOW OFTEN DO YOU WRITE FOR SCHOOL NEWSPAPER 


BD 


15 


BD 


15 


BD 


15 


S001701 


HOW OFTEN DOES TEACHER TALK RE: FOLLCW DIRECTIONS 


BD 


16 


DU 


±0 


BD 


lb 






BW 


23 


BW 


23 


BW 


5 


S001702 


HOW OFTEN DOW TEACHER TALK RE: WRCTE ENOUGH 


BD 


17 


ou 




BD 


1 i 






BW 


24 


BW 


24 


BW 


6 


SOOl* 


HOW OFTEN DOES TEACHER TALK RE: IDEAS IN PAPER 


BD 


18 


BD 


18 


ou 


1 A 






BW 


25 


BW 


25 


BW 


1 


S001704 


HOW OFTEN DOES TEACHER TALK RE: EXPLAIN IN PAPER 


BD 


19 


BD 


1 Q 


OU 








BW 


26 


BW 


26 


BW 


8 


S001705 


HOW OFTEN DOES TEACHER TALK RE: FEELINGS IN PAPER 


BD 


20 


ou 




BD 


5A 






BW 


27 


BW 


27 


BW 


9 


S001706 


HOW OFTEN DOES TEACHER TALK RE: ORGANIZING PAPER 


BD 


21 


BD 




ou 


51 






BW 


28 


BW 


28 


BW 


10 


S001707 


HOW OFTEN DOES TEACHER TALK RE: WORDS IN PAPER 


BD 


22 


BD 


22 


BD 


22 






BW 


29 


BW 


29 


BW 


11 


M A A « A A 

S001706 


HOW OFTEN DOES TEACHER TALK RE: SP, GRAM IN PAPER 


BD 


23 


BD 


23 


BD 


23 






BW 


30 


BW 


30 


BW 


12 


S001709 


HOW OFTEN DOES TEACHER TALK RE: NEATNESS IN PAPER 


BD 


24 


BD 


24 


BD 


24 






3W 


31 


BW 


31 


BW 


13 


S001601 


HOW OFTEN DOES FAMILY WRITE LETTERS TO RELATIVES 


BE 


1 


BE 


1 


BE 


1 


S001802 


HOW OFTEN DOES FAMILY WRITE NOTES OA MESSAGES 


BE 


2 


BE 


2 


BE 


2 


S001803 


HOW OFTEN DOES FAMILY WRITE STORIES OR POEMS 


BE 


3 


BE 


3 


BE 


3 


S001804 


HOW OFTEN DOES FA^u.LY WRITE BUSINESS LETTERS ' 


BE 


4 


BE 


4 


BE 


4 


S001901 


HOW OFTEN DO YOU WRITE A BOOK REPORT 


BE 


5 


BE 


5 


BE 


5 


S001902 


HOW OFTEN DO YOU WRITE ABOUT SCTENCE EXPERIMENT 


BE 


6 


BE 


6 


BE 


6 


S001903 


HOW OFTEN DO YOU WRITE LETTER Tw A RELATIVE 


BE 


7 


BE 


7 


BE 


7 


S001904 


HOW OFTEN DO YOU WRITE NOTES OP. MESSAGES 


BE 


8 


BE 


8 


BE 


8 


S001905 


HOW OFTEN DO YOU WRITE STORIES IHiVT NOT HOMEWORK 


BE 


9 


BE 


9 


BE 


9 


S002001 


WHAT WAS THE LAST THING YOU WROTE IN SCHOOL 


BP 


1 


BF 


1 


BF 


1 






BV 
PS 


14 
2 


BV 


14 


mr 
ov 


1 A 


S002002 


LAST WRITING IN SCHOOL: COPY OVER BEFORE SUBMITING 


BF 


2 


BF 


2 


BF 


4. 






BV 
PB 


.15 
3 


BV 


15 


BV 


15 


S002003 


LAST WRITING IN SCHOOL: MAKE CHANGES BEFORE SUBMIT 


BF 


3 


BF 


3 


BF 


3 






BV 
PB 


16 
4 


BV 


16 


BV 


16 



ERIC 
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riKU> SHORT LAfiEL 

5002004 lAST KRITIMQ IN SCHOOL: MMQS CHANGES AFTER RETURND 

5002005 LAST WRITX199 IN SCHOOL: LIKE DOING THE WRITING 



AGE 9 
BLOCK ITEM 



5/23/86 PAQE 24 
N A E P 

AGE 13 AGE 17 

BLOCK ITEM BLOCK ITEM 



S002101 
S002201 
S002301 
S002302 
S002303 
S002304 
S002305 
S002306 
S002307 
S002501 

S002502 

S002503 

S002504 

S002505 

S002506 

S002701 

S002702 



REPORTS AND ESSAYS WRITTEN FOR SCHOOL LAST 6 WEEKS 
PART OF CLASS TIME SPENT LEARNING TO WRITE REPORTS 
HCM OFTEN ENCOURAGED MAKE NOTES ON TOPIC OF PAPER 
HCM OFTEN ENCOURAGED TO MAKE OUTLINES OF PAPER 
RCW OFTEN DO YOU WRITE PAPER MORE THAN ONCE 
HCW OFTEN DOES TEACHER WRITE SUGGESTIONS ON PAPER 
HCM OFTEN DOSS TEACHER DISCUSS FINISHED PAPERS 
HOW OFTEN DO YOU IMPROVE PAPER AFTER RETURN 
DO YOU ENJOY WORKING ON WRITING ASSIGNMENTS 
HCW OFTEN DOES TEACHER MARK ERRORS ON PAPERS 

HCW OFTEN DOES TEACHER WRITE NOTES ON PAPERS 

HCW OFTEN DOES TEACHER POINT OUT GOOD THINGS 

HCW OFTEN DOES TEACHER POINT OUT NOT GOOD THINGS 

HOW OFTEN DOES TEACHER MAKE SUGGESTIONS FOR NEXT 

HOW OFTEN DOES TEACHER SHOW INTEREST IN WRITItJG 

DID YOU GO TO KINDERGARTEN 

DID YOU GO TO DAY CARE 



50 0270 3 DID YOU GO TO NURSERY SCHOOL 

50 02704 DID YOU GO TO HEADSTART 

S002801 WHERE DID YOU LIVE AT AGE 9 

5002803 WHERE DID YOU LIVE AT AGE 9: CObT^TRY 

5002804 RESIDENCE AT AGE 9 VS. CURRENT RESIDENCE 

5002901 DO YOU USE A COMPUTER AT HOME 

5002902 DO YOU USE A COMPUTER AT THE LIBRARY 

5002903 DO YOU USE A COMPUTER AT A FRIE>n>S HOUSE 



BF 

BV 
PB 

BF 

BV 
PB 

PB 



BG 
BV 



BG 
BV 



BG 
BV 



BG 
BV 



BG 
BV 



BG 
BV 



BH 
PB 



BH 
PB 



BH 
PB 



BH 
PB 



BJ 
PB 



BJ 
PB 



BJ 
PB 



4 

17 
5 

5 
18 
6 



1 
19 

2 
20 

3 
21 

4 
22 

5 
23 

6 
24 

1 
8 

2 
9 

3 
10 

4 
11 



1 
14 

2 
15 

3 
16 



BF 

BV 



BF 

BV 



PB 
PB 
PB 
PB 
PB 
PB 
PB 
PB 
PB 



4 

17 



5 
18 



BG 1 

BV 19 

3G 2 

BV 20 

BG 3 

BV 21 



BG 4 
BV 22 



BG 5 
BV 23 



BG 6 
BV 24 



BH 1 
PB 11 



BH 2 
PB 12 



BH 3 
PB 13 



BH 4 
PB 14 



BH 

BH 

BH 

BJ 1 

PB 17 

BJ 2 

PB 18 

BJ 3 

PB 19 



BF 

BV 



BF 

BV 



PB 
PB 
PB 
PB 
PB 
PB 
PB 
PB 
PB 



4 

17 



5 
18 



bG 1 

BV 19 

BG 2 

BV 20 

BC- 3 

BV 21 

BG 4 

BV 22 



BG 5 
BV 23 



BG 6 
BV 24 



BH 1 
PB 11 



BH 2 
PB 12 



BH 3 
PB 13 



BH 4 
PB 14 



BH 

BH 

BH 

BJ 1 

PB 17 

BJ 2 

PB 18 

BJ 3 

PB 19 



ERIC 
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ZTSM USAGE ACROSS AGES k BVXXS 



PZtLD SHORT LABEL 

S0029L04 aOH OFTEN DC YOU USE A COfTVIER AT SCHOO:. 

5003001 DO YOU USE A COf^FUTCR TO PLAY GAMES 

5003002 DO YOU USE A COMPUTER TO LEARN THINGS 

5003003 DO YOU USE A COMPUTER TO WRITE STORIES OR PAPERS 
S003101 HOW OFTEN DO YOU WRITE OOMFUTER PROGRAMS 

5003201 WHAT DO YOU USUALLY DO AFTER SCHOOL 

5003202 IP YOU GO HOME AFTER SCHOOL, WHO IS USUALLY THERE 
S003301 WHAT KIND OF READER ARE YOU 

S003401 DO YOU EXPECT TO GRADUATE FROM HIGH SCHOOL 

5003501 HOW OFTEN DO YOU READ FOR FUN ON YOUR CMN TIME 

5003502 HGW OFTEN DO YOU TELL A FRIEND ABOUT A GOOD BOOK 

5003503 HCM OFTEN DO YOU TAKE BOOKS OUT OF THE LIBRARY 

5003504 HCM OFTEN DO YOU SPEND YOUR OWN MONEY ON BOOKS 

5003505 HCM OFTEN DO YOU READ BOOK BASED ON NOVIF. YOU SAW 

5003506 HCM OFTEN DO YOU READ BOOKS BY AN AUTHOR YOU LIKE 

5003601 HOW OFTEN DO YOU GO TO A MOVIE 

5003602 HCM OFTEN DO YOU GO TO A PLAY 

5003603 HOW OFTEN DO YOU GO TO A CONCERT 

5003604 HOW OFTEN DO YOU GO TO A PARTY 

5003605 HCM OFTEN DO YOU GO TO THE PUBLIC LIBRARY 

5003606 HOW OFTEN DO YOU TRAVEL TO A PLAv.fi ^'-ZAY FROM HOME 

5003607 HOW OFTEN DO YOU GO SHOPPING 

5003608 HCM OFTEN DO YOU GO TO A SPORTS EVciTT 

5003609 HCM OFTEN DO YOU PLAY CARD OR TABLE GAMES 

5003610 HOW OFTEN DO YOU VISIT RELATIVES 

5003611 HCM OFTEN DO YOU GO TO A MUSEUM 

5003612 HOW OFTEN DO YOU CAMPING 

5003613 HOW OFTEN DO YOU STAY HOME ALONE 



AGE 9 

BLOCK ITEM 

BJ 4 

PB 17 

3J 5 

PB 18 

BJ 6 

PB 19 

BJ 7 

PB 20 

BJ e 

PB 21 

BJ 9 

PB 23 

BJ 10 

PB 23 

BK 1 

BV 28 



BK 
PB 



RK 
BV 



BK 
BV 



BK 



BL 
BL 
BL 
BL 
BL 
BL 
BL 
BL 
BL 



BV lU 



BK 6 
BV 11 



BK 7 

BV 12 

BK 8 

BV 13 



BL 10 

BL 11 

BL 12 

BL 13 



AGE 
BLOCK 

BJ 
PB 

BJ 
PB 

BJ 
PB 

BJ 
PB 

BJ 
PB 

BJ 
PB 

BJ 
PB 

BK 
BV 

BK 
PB 

RK 
BV 

BK 
BV 

BK 
BV 

BK 
BV 

BK 
BV 

BK 
BV 

BL 

BL 

BL 

BL 

BL 

BL 

BL 

3L 

BL 

BL 

BL 

BL 

BL 
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ITEM BLOCK 

4 BJ 
20 PB 

5 BJ 
21 

6 BO 

22 PB 

7 BJ 

23 PB 

8 BJ 

24 PB 



PAGE 25 



17 

ITEM 



9 
25 



10 PB 
26 



1 
28 



7 
12 

8 
13 



BK 



2 BK 
10 PB 



RK 
BV 



BK 
BV 



5 BK 

10 BV 

6 BK 

11 BV 



BK 
BV 



BK 
BV 



1 BL 

2 BL 

3 BL 

4 BL 

5 BL 

6 BL 

7 BL 

8 BL 

9 BL 
10 BL 
V BL 

12 BL 

13 BL 



4 

20 



5 
21 



6 
22 



7 
23 



8 
24 



PB 25 



26 



2 
10 

8 

4 

9 

5 
10 

6 
11 

7 
12 

6 
13 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 
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hrVtJXDXX 2 

ZTCH USMC ACROSS AOCS k BIOCKS 



FXBXi) SHORT ZAaCL 

S003614 1«AT ACnviTY DO YOU DO MOST OFTEN 
S003701 DO YOU EVER TZEL BORED AT SC3KK5L 

5003801 IXJRIiW PAST YEAR HC*f OrTEN SEWT TO PRINCIPALS OFF 

5003802 DURHW PAST YEAR HW OFTEN PLACED ON PROBATION 

5003803 EURIW3 PAST YEAR HCW OFTEN GIVEN DETENHON 
DURIIW PAST YEAR HCM OFTEN WARNED ABOUT ATTENDANCE 
DORHW PAST YEAR HOW OFTEN WARNED ABOUT GRADES 
DURING PAST YEAR HCW OFTEN WARNED ABOUT llEHAVIOR 
HCW MANY OLDER BROTHERS AND SISTERS 

S003902 !;0W MANY YOUNGER BROTHERS AND SISTERS 
S004001 HOW MANY DAYS OF SCHOOL MISSED LAST MOOTi: 
HOW MANY TIMES LATE FOR SCHOOL LAST MONTH 



S003804 
S003805 
S0n3806 
S0039n 



S004101 



S004201 

S004202 

S004203 

S004204 

S004205 

S004206 

S004207 

S004208 

S004209 

S004210 

S004211 

S004301 

S004302 

S004303 

S004304 

S004305 

S004306 

S004307 



HCW OFTEN READING: HELPS ME DECIDE WANT TO BE 

HCW OFTEN READING: HELPS ME LEARN TO FIX THINGS 

HCW OFTEN READING: HELPS UNDERSTAND PEOPLES ACTION 

HOW OFTEN READING: READING IS IMPORTATrT 

HOW OFTEN READING: BETTER FEWER HARD WORDS 

HCW OFTEN READING: BETTER F^WER LONG SENTENCES 

«rw OFTEN READING: BhTTER If IT MATTERED TO ME 

HOW OFTEN READINGS BETTER IF TEACH GAVE MORE TIME 

HCW OFTEN READING: BETTER IF DIDNT HAVE SO MUCH 

HOW OFTEN READING: BETTER IF WASNT TESTED ON IT 

HOW OFTEK READING: LIKE MORE IF COULD TALK W OTHER 

HCW OFTEN DO YOU READ A STORY OR NOVEL 

HOW OFTEN DO YOU READ A POEM 

HOW OFTEN DO YOU READ A PLAY 

HOW OFTEN DO YOU READ A NEWSPAPER 

KOW OFTEN DO YOU READ A MAGAZINE 

HOW OFTEN CO YOU READ A SCIENCE BOOK 

HOW OFTEN DO YOU READ A BIOGRAPHY 
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7 
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APPCIDIX 2 

ZTDf USM3E AOtOSS AGES k 



rZSLD SHORT LABEL 

3004308 HCm OFTEN DO YOU READ A HOH-TO-DO BOOK 

3004309 HCW OFTCr DO YOU READ A BOOK ABOUT OIIIEH TIKES 
S004310 HOW OFTEN DO YOU READ A SPORTS GOOK 

3004311 HG^ OFTEN tX) YOU READ WRD6 OF A SOnS 

3004401 HOH OFTEN DOES SC«fEOI9E READ AZXXJD TO YOU 

3004401 HOW OFTEN DO YOU READ ALOUD TO SOMEONE 

3004501 Hcm OFTEN DOES FAMILY READ NEHSPAFERS 

3004502 mm OFTEN DOES FAMILY READ MAGAZINES 

3004503 HCM OFTEN DOES TAMILY READ BOOKS 
S004504 HCm OFTEN DOES FAMILY HEAD RECIPES 

5004601 HOW OFTEN WITH NEW READING TEACHER POINT HARD WORD 

5004602 HOW OFTEN WITH NEW READING TEACHER PREVIEW READING 

5004603 HOW OFTEN WITH NEW READING TEACHER READ PART ALOUD 

5004701 HOW OFTEN DOES TEACHER LIST OF QUESTS AS YOU READ 

5004702 HCW OFTEN DOES TEACHER TELL HCM TO FIND MAIN IDEA 

5004703 HOW OFTEN DOES TEACHER TELL HOW TO READ FASTER 

5004801 HOW OFTEN TB'JE: WRITING HELPS ME GET A GOOD JOE 

5004802 HCW OFTEN TRUE: WRITING HEL^^ ME SHARE MY IDEAS 

5004803 HCW OFTEN TRUEj WRITING HELPS SKO< I KNOW THINGS 

5004804 HOW OFTEN aJE: WRITING HELPS KEEP IN TOUCH FRIEN 
S005001 WHEN FREE TIME. HCW OFTEN WATCH TV 

3005002 WHEN FRE2 TIME, HCW OFTEN READ A DOCK 

.05003 WHEN FREE TIME, HCW OF*^ W iTE IN DIARY 

5005004 WHEN FREE TIME, HCW OFTEN CALL A FRIEND 

5005005 WHEN FREE TIME, HOW OFTEN BE WITH FRIENDS 
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AGE 13 


AQE 


17 


PZELO 


SHORT LABEL 
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ITEM 


BLOCK 


ITEM 
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laiKM FRKE TTMK HflV nFTTM nn GUTiOVTWl 
rwi^a r<uM» ixnby ixv/n urxui vaj sscivirt'irAj 


BS 


o 


BS 


6 


BS 


6 






BW 


6 


BW 


6 






S005007 


VHKN WWt TTMr nru r\i<fi*mtf DT^v X cn/yom 
mi&n rxuui AxnCfy m/ff Ur iUf ItmAX A S<\/Kr 


BS 


7 


BS 


7 


BS 
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BW 


7 


BW 
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vnu*n Ftittb nnxtf nuw uften vjO hui^InG or rlSHlNG 


ES 


8 


BS 
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BW 


8 


BW 


8 








MHILig Vngnp tTTUr UTAf ft WfM fP&tTP ^ U^tv 

vm&ii riubc* AxnCff iiuTf urisn aAKC a vwl>.«\ 


BS 


9 


BS 


9 


BS 


9 






BK 


9 


BW 


9 






5005010 




BS 


1 n 
IQ 


BS 


10 


BS 


10 






BW 


10 


BW 


10 






S005011 




BS 


11 


BS 


11 


BS 


11 






BW 


11 


bW 


11 






S005012 


mun rnbA ixnit^ Rv/V urTEtH K£AU A nEWSPAFER 


BS 


12 


BS 


12 


BS 


12 






BW 


12 


UW 


12 








tmE*n riuuci xinbi orT£R G£T A ohaux 


BS 


13 


BS 


13 


BS 


13 






BW 


13 


BW 


13 






SQQ5014 




BS 


14 


BS 


14 


BS 


14 






BW 


14 


BW 


14 






S005015 


WHEN FPRE TTMP W1U ftlFTPM UDTTP K T iiwiL' J 


BS 


15 


BS 


15 


BS 


15 






BW 


15 


BW 


15 






S005016 


WtffiW FSEC TTMK HOW OFTPff TTCPTTN Tn MrCTr 


BS 


16 


BS 


16 


BS 


16 






BW 


16 


BW 


16 






3005017 


finch.* riuub xxnCf nwv urxbn iaj bUfiLrnirKs eijSL 


BS 


17 


BS 


17 


BS 


17 






BW 


17 


BW 


17 






S005018 


WHEN FREE TIME, iX/ SCJifiTHING ELSE, WHAT IS IT 


BS 


17 


BS 


17 


BS 


17 






BW 


17 


BW 


17 






c:on50i Q 


UlfVW vow T*Tlyft* Utl&ft ii ■ nnj »n*Mw\ a i~ m tnvAMi 

vm&n riUUb irJUUi rItUvr ACXIVITX SPEXu) nOST TInE 


BS 


18 


BS 


18 


BS 


18 






BW 


16 


BW 


18 






S005101 


HOW OFTEN WHEN STUDY FOR TEST: READ OVER MATERIAL 


BT 


1 


BT 


1 


BT 


1 


S005102 


HOW OFTEN WHEN STUDY FOR TEST: TAKE tWTES ON READ 


Bl- 


2 


BT 


2 


BT 


2 


S005103 


HOW OFTEN WHEN STUDY FOR TEST: MMCE OUTLINES 


BT 


3 


Bl 




BT 


3 


S005104 


HOW OFTEN WHEN STUDY FOR TEST: QUES IN TEXTBOOK 


BT 


4 


BT 


4 


BT 


4 


S005105 


HOW OFTEN WHEN STUDY FOR TEST: ANSWER •:WN QUESTNS 


BT 


5 


BT 


5 


BT 


5 


S005106 


HCW OFTEN WHEN STUDY FOR TEST: QUESTION OTHERS 


BT 


6 


BT 


6 


BT 


6 


S005201 


HOW OFTEN nn vntf PRXD AT/Mm TM c/miv^r 


BT 


7 


BT 


7 


BT 


7 


S005202 


HOW OFTEN DO YOU READ ON YOUR OWN IN SCHOOL 


BT 


8 


BT 




BT 


8 


S005203 


HOW OFTEN DO YOU WORK IN A WORKBOOK 


BT 


9 


BT 


9 


BT 


9 


5005301 


nv/n urxuv viv t\f ijX*34\A<\X xv aSinJJ UN vnr* 


3T 


10 


BT 


10 


BT 


10 






BW 


32 


BW 


32 


BW 


14 


S005302 


HOW OFTEN GO TO LIBRARY TO LOOK UP FACT FOR SCHOOL 


BT 


11 


BT 


11 


BT 


11 






BW 


33 


BW 


33 


BW 


15 


S005303 


HOW OFTEN GO LIBTARY TO FIND BOOKS FOR HOBBIES 


BT 


12 


BT 


12 


BT 


12 






BW 


.^4 


BW 


34 


BW 


16 


S005304 


HOW OFTEN GO TO LIBRAE/ FOR QUIET PLACE TO READ 


BT 


13 


BT 


13 


BT 


13 






BW 


35 


BW 


35 


BW 


17 



ERIC 



Appem>xx 2 

ITEM USKSE ACROSS AGES 4 BLOCKS 



b/23/96 
N A E P 



PAGE 29 



FIBU) 


SBORT LABEL 


AGE 9 
BLOCK ITEM 


AGE li 
BLOCK ITEM 


AGE 
BLOCK 


17 
TTEt 


S005305 


HOW OFTEN GO TO LZBRAR3f TO TAKE OUT BOOKS 


BT 


14 


BT 


14 


BT 


14 






BW 


36 


BW 


36 


BW 


18 


S005401 


HOW OFTEN 00 YOU HATCH NEMS OUt TELEVISION 


BT 


15 


3T 


15 


BT 


15 






BW 


19 


BW 


19 


TM 


1 


SC05402 


HOW OFTEN DO YOU READ A NEWS MAGAZINE 


BT 


16 


BT 


16 


BT 


16 






BW 


20 


BW 


20 


Bi4 


2 


S005403 


HCM OFTEN DO YOU SIEAD NEWSPA*^ NOT COMICS OR SPRT 


BT 


17 


BT 


17 


3T 


17 






BW 


21 


BW 


21 


BW 


3 


S005404 


HOW OPTBT: DD you listen TO NEWS ON RADIO 


BT 


18 


BT 


18 


BT 


18 






BW 


22 


BW 


22 


BW 


4 


S005701 


HAVE YOU APPLIED FOR ADMISSION TO A COLLEGE OR UNV 










BJ 


10 


S005702 


HAVE YOU APPLIED TO A FOUR-YEAR COLLEGE 










BJ 


10 


S005703 


HAVE YOU APPLIED TO A TWO-YKAR COLLEGE 










EJ 


10 


S005704 


HAVE YOU APPLIED TO OTHER COLLEGE OR UNIVERSITY 










BJ 


10 


S005705 


HAVE YOU APPLIED; SPECIFY OTHER COLLEGE OR UNIV 










BJ 


10 


soosaol 


WHAT ARE YOUR LONG-TERN CAREER GOALS 










BJ 


11 


S005802 


LONQ-TERH CAREER GOAL CODE 










BJ 


11 


S005803 


ICaa TERM CAREER GOAL: NON-CODEABLE 










BJ 


11 


S005901 


WHERE DID YOU LITE AT AGE 13 










BH 


6 


S005903 


WHERE DID YOU LIVE AT AGE 13: COUNTRY 










BH 


6 


S005904 


RESIDENCE AT AGE 13 VS. CURRENT RESIDENCE 










BH 


6 


S006001 


WHICH COURSES HAVE YOU TAKENrGENERAL SCIENCE 










BL 


22 


S006002 


WHICH COURSES HAVE YOU TAKEN: BIOLOGY 










BL 


23 


S006003 


WHICH COURSES HAVE YOU TAKEN: CHEMISTRY 










6L 


24 


S006004 


WHICH COURSES HAVE YOU TAKBN:PHYSICS 










BL 


25 


S006006 


WHICH COURSES HAVE YOU TAKEN: SPEC OTH SCIENCE (1) 










BL 


26 


S006008 


WHICH COURSES HAVE YOU TAKEN:SPEC OTH SCIENCE (2) 










BL 


26 


S006010 


••;ICH COURSES HAVE YOU TAKEN: SPEC OTH SCIENCE (3) 










BL 


26 


S006101 


WHICH COURSES ':>*T5 YOU TAKEN: GENERAL MATH 1 










BN 


12 


S006102 


WHICH COURSES HATE YOU TAKEN:GENERAL MATH 2 










BN 


13 


S006103 


WHICH COURSES HAVE YOU TAKEN: FIRST YEAR AIjGEBRA 










BN 


14 


S006104 


WHICH COURSES HAVE YOU TAKEN: SECOND YEAR ALGEBRA 










BN 


15 


S006105 


WHICH COURSES HAVE YOU TAKEN: GEOMETRY 










BN 


1€ 


S006106 


WH*CH COURSES HAVE YOU TAKEN: CALCULUS 










BN 


17 


S006108 


WHICH a>URSES HAVE YOU TAKEII: SPECIFY OTHER MATH(l) 










BN 


18 


SODSllO 


WHICH COURSES HAVE YOU TAKEN:SPECIFY OTHER MATH(2) 










BN 


19 


S006112 


WHICH COURSES HAVE YOU TAKEN: SPECIFY OTHER MATH(3) 










BN 


20 


S006201 


RATE YOUR SCHOOJ> IN: PREPARING STUDENTS FOR COLLEGE 










BP 


7 
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APPeRDZX 2 

XTCM USMS ACROSS AGES k BLOCKS 



rXBLD SKORT LABEL 

S0062C2 RATE YOUR SCHOOL IR:FR£PARIMQ STUDENTS FOR CAREER 

5006203 RATE TOUR SCHOOL IN:PR£I>ARI1IQ STUDEITTS FOR L.TPE 

5006204 RATE YOUR SCHOOL IN:VARIETY OF EXTRACUR ACTIVITIES 

5006205 RATE YOUR SCHOOL IN:QUALITY OF EXTRACUR ACTIVITES 

5006206 RATE YOUR SCHOOL INlFACULTY DTTEREST IN STUI^SNTS 

5006207 RATE YOUR SCHOOL HllQUALITy OF " ^CULTY 
S00620S RATE YOUR SCHOOL INlQUALITY 0. JTUDEZTT LIFE 

5006301 SOwOL EXPERTSNCES:! AH SATSFIED WITH MY EDUCATION 

5006302 SCHOOL EXFERIE!ICES:MOT ZX9dJSn WHAT I NEED TO KNOW 

5006303 SCHOOL EXPERIENCES: HAVE KM; DISCIPUtE FRBS THIS YR 

5006304 SCHOOL EXPERIENCES:I AH UTTERESTED IN SCHOOL 

5006305 SCHOOL EXPERIEZ9CES:0NCS IN A WHILE I CUT CLASS 

5006306 SCHOOL EXPERIENCES : I DON'T FEEL SATE AT THIS SCHL 

5006307 SCHOOL EXPERIENCES : WISH I COULD GO TO DIFF SCHOOL 
S006401 HAVE YOU EVER BEEN IN PROGRAM: REMEDIAL ENGLISH 

SO 06402 HAVE YOU EVER BEEN IN PROGRAM: REMEDIAL MATH 

5006403 HAVE YOU EVER BEEN IN PROGRAM: HONORS ENGLISH 

5006404 HAVE YOU ETET* BEEN IN PROGRAM: HONORS MATHEMATICS 

5006405 HAVE YOU EVER BEEN IN PROGRAM :HCaWRS SCIENCE 

5006406 (lAVF YOU EVER BEEN IN PROGRAM: BILINGUAL PROGRAM 

5006407 HAVE YOU EVER BEEN IN PROQRAM:PAMILY-LIFE,SCX ED 

5006408 HAVE YOU EVER BEEN IN PROGRAM'ALCOHL, DRUG-ABUSE ED 

5006409 HAVE YOU EVER BEEN IN PROGRAM:SPEC PHYSICAL PROGRM 

5006410 HAVE YOU EVER BEEN IN PROGRAM:SPEC SPEECH PROGRAM 
S006501 HAVE YOU TAKEN COURSES : AGRICULTURE, INCLD HORTICULT 
S0P6502 HAVE YOU TAKEN O0URSES:AUT0 MECHAf^CS 

5006503 HAVE YOU TAKEN COURSES: COMMERCIAL ARTS 

5006504 HAVE YOU TAKEN COURSES: COMPUTER PROGRAMMING 

5006505 HAVE YOU TAKEN COURSES: CONSTRUCTION, CARPENTRY TRDS 

5006506 HAVE YOU TAKEN COURSES : CONSTRUCTION TRADES : ELECTRL 

5006507 HAVE YOU TAKEN COURSES :a»rrRUCTION TRADES: MASONRY 

5006508 HAVE YOU TAKEN COURSES :CONSTRUCTI'^N TRADES :PLirnNG 

5006509 HAVE YOU TAKEN COUP3ES:CQSMETOIjOGY,HAIRDRESSING 

5006510 HAVE YOU TAKEN COURSES : DRAFTING 

5006511 HAVE YOU TAKEN COURSES: ELECTRONIC 

50 06512 HAVE YOU TAKEN COURSES : 'JOME EC, DIETETICS, CHILD CAR 
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APPSMDIX 2 

ZTBK USJySS ACROSS AGES 6 BLOCKS 



PZKLC SHOBT tABSL 

5006513 HAV£ YOU TATEN OOURSBS : MACHINE SHOP 

5006514 HAVE YOU TAKEN COURSES :MBDICAL OR DEtlTAL ASSIStNT 

5006515 HAVE YOU TAKEN COURSES: PRACTICAL NURSINS 

5006516 HAVE YOU TAKEN COURSES:FOOD SERVICE OCCUPATIONS 

5006517 HAVE YOU TAKEN COURSES :SALES OR KSRCHANDISINS 
S0U6518 HAVE YOU TAKEN COURSES ^SECRETARIALrOPFlO; VIORK 

5006519 HAVE YOU TAKEN COURSES iMSLDINS 

5006520 HAVE YOU TAKEN C0URS:3;0THER 

5006601 yma take msr, or your time year after high school 

5006602 TAKE MOST OP TIME AFTER HIGH SCH00L:SPEClFy OTHCH 

5006701 OTHER PiANS FOR YEAR AFTER HIGH SCHOOL :Vi0RK 

5006702 OTHER PLANS FOR I£AR AFTER HIGH SCHOOL :APFREOTICC 

5006703 aXHER PLANS FOR YEAR AFTER HIGH SCHOOL: MILITARY 

5006704 OTHER PLANS FOR YEAR AlfTSR Him! SCHOOL iKQMEMAKER 

5006705 OTHER PUw'S FOR YEAR AFTER HIGH SCHOOLlVOC SCHOOL 

5006706 OTHER PLANS FOR YEAR AFTER HIGH SCHOOL :COKM COLLEG 

5006707 OTHER PLANS FOR YEAR AFTER HIGH SCHOOL:VOC COURSES 
S006706 OTHER PLANS FOR YEAR AFTER HIGH SCHOOL :4-YR COLLEG 

5006709 OTHER PLANS FOR YEAR AFTER HIGH SCHOOL:TRAVEL,NONE 

5006710 OTHER PLANS FOR YEAR AT.'ER HIGH SCHOOL: ii-w" OTHER 
S006801 HOW MUCH FREE TIME ON AVERAGE SCHOOL DAY 



AGE 9 
BLOCK ITEM 



AGE 13 
BLOCK ITEM 
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AGE 17 
BLOCK ITEM 

BK 31 

BK 32 

BK 33 

BW 34 

Bia 35 

BK 36 

BW 37 

BW 38 



BJ 11 
PB 24 
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BY 
BY 
BY 
BY 
BY 
BY 
BY 
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NATIONAL ASSESSMENT OF EDUCATTONAL PROGRESS 



GLOSSARY 



administration, T*;e conduct of a 
National Assessment session. 



Administration Schedule. A list of the 
name, age and sex of each student 
invited to a particular assessment 
session. 



administration time. The total time 
allowed for an item. (Includes the 
time allowed for the stimulus and the 
response.) 

administration timetable. Time periods 
during the school year when the 
various grade/age groups are assessed. 
The time periods for the Year 15 
assessment were: 

Grade 8/Age 13 

October 10 to December 16. 1983 

Grade 4/Age 9 
January 2 to March 9. 1984 

Grade 11 /Age 17 

March 12 to May 11. 1984 



administrative units. Geographic areas 
such ^ states, counties, school 
distri.vo. etc. 



AERA. American Educational Research 
Association. 



age-eligible. An individual who meets 
the age definition for one of the 
National Assessment populations: 
9-year-olds. 13-year-olds, 
17-year-olds. 



aggregate estimate. Estimate fo'' a 
combination of smaller groups for 
which estimates have been produced. 



allocation. Apportionment of a total 
sample size to various parts of the 
population (See final allocation.) 



almanacs. The sets of tables 
summarizing NAEP results. 



anchormg. The process of 

characterizing score levels in terms 
of predicted observable behavior. 



ARM, See Average Response Method. 



assessment. The documentation of the 
progress in knowledge, skills and 
attitudes of American youth. Measures 
are taken at periodic intervals for 
each learning area, with the goal of 
determining trends and reporting the 
flndings to the public and to the 
education community. See also 
National Assessment of Educational 
Progress, 
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assessment administrator. Individual 

employed to administer the assessment 
in participating schools. 



dfLOG. A computer program for 

estimating item parameters by marginal 
estimation procedures. 



assesshient session. The period of time 
during which a NAEP package is 
administered to one or more 
individuals. 



Average Response Method. A 

regress-based technique for predicting 
for a respondent the conditional 
distribution of an average score on a 
set of exercises given responses to at 
least one of the exercises and other 
information. Used to produce the NAEP 
Year 15 Writing Scale. 

average sample size, V t average sample 
obtained per sampling unit selected. 

background and attitude items. See 
non-cognitive assessment, 

bias. In statistics, the difference 
between the expected value of an 
estiniator and the population parameter 
being estimated. If the average value 
of the estimator over all possible 
samples (the estimator's expected 
value) equals the parameter being 
estimated*, the estimator is said to be 
unbiased; otherwise, the estimator is 
biased. 



B!B (Balanced Incomplete Block) 

spiralling. A complex variant of 
multiple matrix sampling, in which a 
small subset of 'terns is administered 
to each respondent in such a way that 
each pair of items is administered to 
a nationally representative subsample 
of respondents. 



block. A group of assessment items 
created by dividing the item pool for 
a grade/age into subsets. Used in the 
implementation of the BIB and UBIB 
Spiral sample design. 

booklet. The assessment instrument 
created by combining blocks of 
assessment items. 



bridging. An administration of the 
same set of exercises under two 
different conditions or to two 
different populations to allow a 
statistical link ("bridge") to be 
established between results under the 
duTerent circumstances. 



calibrate. To estimate the param Jers 
of a set of items from respon ,es of a 
sample of a «et of examinees. 



category (scoring). A classification of 

a response to an open-ended item. See 
Scoring Guide, 



category within a variable. A 

sub-classifiration within a variable, 
or subgroup. For example, Male ana 
Female are categories of the subgroup 
iex. See Reporting Subgroups. 

cell. The smallest unit of a table. 

For example, a two-way table with 5 
rows and 7 columns contains 35 cells 
(5x7 = 35). 
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census tract (CT). Small, relatively 
permanent areas into which large 
cities and adjacent areas are divided 
for the purpose of providing 
small-area statistics. The average 
census tract contains approximately 
4,000 residents. 



clw'tering. The process of form'-ng 
groups of sampling units. 



conditioning variables. Demographic 
variables characteriiring respondent. 
Used in construction of plausible 
values. 



controlled selection, A method of 
probability sampling involving 
balanced samples on asymmetrical 
controls. Further controls beyond 
stratification are used. 



codebooL A printout of the raw data 
files tor each student, excluded 
student, teacher and school in a 
particular grade/age. 



coefficient of variation. The ratio of 
the standard deviation of an estimate 
to the value of the estimate. 



cognitive assessment. The portion of the 
Year 15 NAEP whicK assessed students' 
abilities in the learning areas of 
reading and writing. 

combined ratio estimator. The ratio 
estimator resulting from first 
estimating the numerator and the 
denominator values and then using the 
quotient of these as the estimate of 
the ratio. 



common block. A group of background 
items included in the beginning of 
every assessment booklet. 



complete enumeration survey. Survey in 
which the entire populrtion is 
enumerated or surveyed: a census. 



conditional probability. Probability of 
an event, given the occurrence of 
another event. 



CPS. See Current Population Survey. 



Current Population Survey. A household 
sample survey conducted monthly by the 
Bureau of the Census to provide 
estimates of employment, unemployment, 
and other characteristics of the 
general labor force, of the population 
as a whole, and of various subgroups 
of the population. 



CV. See coefficient of variation. 



data edifing. The process by which 
assessment responses and other 
information are verified. 



data entry. The process by which 
assessment responses and other 
information are transferred from paper 
to computer. 

degrees of freedom. [of a variance 

estimator]. The number of independent 
pieces of information used to generate 
the estimate. For Year 15 NAEP, this 
is at most 32. the number of PSU 
pairs. 



demographic subgroups. See reporting 
subgroups. 
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derived variables. Subgroup data that 
were not obtained directly from 
assessment responses, but through 
procedures of interpretation, 
classification or calculation. See 
also reporting subgroups. 



design effects. The ratio of the 

variance accounting for the sample 
design to a conventional variance 
estimate which assumes a simple random 
sample. 



distractor. An incorrect response 

choice included in a multiple-choice 
exercise. 



entry mode. Processing option under the 
data entry system; used for the 
initial transcription of assessment 
data. 



ETS. See Educational Testing Service. 



examinee. Same as respondent. 



Excluded Student Questionnaire. An 
instrument used in the Year 15 
assessment; completed for every 
student who was sampled but was 
ineligible for or excluded from the 
assessment. 



District Supervisor. One of 16 

supervisors responsible for contacting 
schools, arranging and conducting 
introductory meetings, recruiting, 
training and providing support to 
Exercise Administrators, distributing 
and collecting questionnaires, 
completing administrative reporting 
forms* and pr eking and shipping all 
materials to HTS. 



double-length block. A group of 

assessment exercises, 28 minutes long, 
created to accommodate the use of 
longer exercises; used in UBIB spiral 
administration. 



EA. See Exercise Administrator. 



ECS. See Education Commission of the 
States. 



Education Commission of the States. 

The NAEP grantee prior to Year 15. 



Educational Testing Service. The NAEP 
grantee for Year 15. 



excluded students. Sample students who 
were determined by the school to 
unable to paticipate because they had 
limited English-speaking ability, were 
educable mentally retarded, or 
functionally disabled: also sample 
students who were no longer in the 
school, or were not age- or 
grade-eligible. 



Exercise Administrator. The person 
whose primary function was to 
administer the assessment booklets to 
the sample students. 



exercise. A task designed to measure an 
objective. Because NAEP does not 
administer "tests," but instead 
describes educational achievement over 
time, the term "exercise" is often 
used mstead of the term "item" or 
"test item." The terms "item" and 
"exercise" are used synonymously in 
this report. 



exercise booklet. See booklet. 
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exercise part. Each portion of an 
exercise that asks a separate 
question.' Parts may all pertain ^o 
one stimulus, such as a graph or a 
table, or may concern the same topic. 

exercise pool. The entire set of 
exercises prepared for a learning 
area. This set includes recycled 
exercises developed for previous 
assessments but not used due to 
exercise booklet or budgetary 
constraints and newly developed 
exercises. 



expected value. The average of the 

sample estimates given by an estimator 
over all possible samples. If the 
estimator is unbiased, then its 
expected value will cqua* the 
population value being estimated. 



extra subsampling. Subsampling of 
segments which results in less than 
one Seconda»7 Sampling Unit being 
surveyed. Used in segments with a 
large amount of recent growth in 
population, such as a newly 
constructed apartment house or trailer 
park. 



field test. A pretest of exercises to 

obtain information regarding clarity, 
difficulty levels, timing, feasibility 
and special administrative problems 
needed for revision and selection of 
exercises to be used in the 
assessment. 



final allocation. Usually determined by 
rounding or adjusting a preliminary 
sample allocation to integer numbers 
See allocation. 



fi)ils. The correct and incorrect 
response choices included in a 
multiple-choice exercise. 



fourth-stage sampling unit. See 
multi-stage sample design. 



free-response item. Same as open-ended 
response item. 



grade-eligible. An individual who meets 
the grade definition for one of the 
Year 15 National Assessment 
populations: Grade 4. Grade 8. or 
Grade 1 1 . 



grade /age-eligible. An individual who 
meets the age or grade definition for 
one of the Year 15 National Assessment 
populations: Grade 4 or Age 9. Grade 
8 or Age 13. Grade 1 1 or \ge 17. 

group administered package. A package 
containing exercises which can be 
administered to groups of students. 



group effect. The difference between 

the mean for a group and the mean for 
the nation. 



holistic scoring. A method of scoring 
open-ended response exercises that 
evaluates a response on the basis of 
overall impression. 



imputation Prediction of a missing 
value according to some procedure, 
using a mathematical model in 
combination w*'h available 
information. See plausibie values. 



first stage sampling unit. See 
multi-stage scunpk design. 
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imputed race/ethnicity. The 

racc/ethnicity of an assessed student* 
as derived from his or her responses 
to three particular common background 
items. A Year 15 reporting subgroup. 

in-school sample design. Sample design 
for the National Assessment school 
survey. Sec sample design. 

individual completion rate. Proportion 
of cligibles in the sample who recpoiid 
by completing one or more assessment 
packages. 

ineligible. Person who is not eligible 
for National Assessment because he or 
she does not satisfy grade or age 
requirements (sec grade/age-eligible). 
Also includes grade/age eligibles who 
are either mentally or physically 
handicapped so that they cannc t 
respond to the exercises administered, 
non-English-speaking, incarcerated, or 
non-readers. 



informative writing. A writing 

objective of the Year 15 assessment: 
writing that is used to share 
knov/Iedge and convey messages, 
instructions and ideas. 



inieHigem data entry system. A set 

of computer programs and procedures 
developed in accord^ince with the NAEP 
de;ign to validate, verify, transcribe 
and check for the reasonableness of 
available data. 



IRT. See item rcsi^onse theory, 
item. See exercise. 



item booklet. See booklet. 



item part. See exercise part. 



item pool. See exercise pool. 



item response theory. Test analysis 

procedures that assume a maethematical 
model for the probability that given 
examinee will respond correctly to a 
given exercise. 



jackbuje. A procedure to estimate 
standard errors of percentages and 
other statistics. Particularly suited 
to complex sample designs. 

learning area. One of the areas 

assessed by National Assessment, e.g.. 
art, career and occupational 
development, citizenship, literature, 
mathematics, music, readin;^. science, 
social studies and writing. 



literary writing. In the Year 15 

assessment, writing from a basis of 
experience and im^^jinative ideas to 
share experiences and understand the 
world. 



LOGIST. A computer program for 
es^'tnating item parameters by joint 
estimation procedures. 



machine -readable catalog. Year 15 
computer processing control 
information, IRT parameters, foil 
codes and labels in a 
computer-readable format. 



item block. See block. 



major strata. Used to stratify the 

primary sampling frame within each 
region. Involves stratification by 
size of cominunity and degree of 
ruralness (SDOC). 



marc'nal value, A row or column total, 
the sum of all cell values in the row 
or column. 



multi-stage sample design. Indicates 

more than one stage of sampling. An 
exanr-ple of four-stage sampling: 1) 
sample of counties (primary sampling 
units or PSUs): 2) sample of blocks 
within each sample county; 3) sample 
of housing units within each sample 
block; 4) sample of grade/age- 
eligibles within each sample housing 
unit. 



meanparts estimator. Estimates a 

subgroup average score across a set of 
items by the average of the subgroup 
scores for each of the items. Can be 
extended to any linear estimator. 



mechanics scoring. A method of scoring 
open-ended response exercises that 
evaluates elements of sentence 
construction^ word choice, spelling, 
punctuation and capitalization. 



multiple matrix sampling. Sampling 
plan in which different samples of 
respondents take different samples of 
items. 



multiple<ounty PSU, A primary sampling 
unit (PSIJ) composed of two or more 
counties. 



NIE, Natio..al Institute of Education. 



modal age. The age of the majority of a 
group of grade-eligible students: Age 
9 for fourth graders. Age 13 for 
eighth graders and Age 17 for eleventh 
graders. 



modal grade. The grade attended by the 
majority of a group of age-eligible 
students: the fourth grade for 
9-ycar-olds, the eighth grade for 
13-year-olds and the eleventh grade 
for 17-year-olds. 



mode of administration. The method by 
which students are administered 
assessment instruments; in Year 15 the 
modes of administration were spiralled 
and taped. 



nine-year-olds. One of the National 
Assessment target populations. For 
Year 15, defined as persons born 
during calendar year 1974. 



non-cognitive assessment. The 

background questions used to collect 
information from students about 
activities, attitudes and 
demographics. 



nonresponse. The failure to obtain 
responses or measurements for all 
sample elements. 



nonsampiing error. A general term 
applying to all sources of error 
except sampling error. Includes 
errors from defects in the sampling 
frame, response or measurement error, 
and mistakes in processing the data. 



objective, A desirable education goal 
agreed upon by scholars in the field, 
educators and concerned lay persons, 
and established through the consensus 
approach. 



objectives re-development. A review of 
the learning area objectives following 
the initial assessment of a learning 
area; carried out by scholars in the 
field, educators and concerned lay 
persons. May result in revision, 
modification or total rewriting of the 
learning-area objectives to reflect 
current curricular goals and emphases. 

observational unit. The individual 
units for which characteristics are 
observed '.t nrieasurements are obtained. 



observed race/ethnicity. Race/ethnicity 
of an assessed as perceived by the 
Exercise Administrator. 



paced tape, A tape recording 

accompanying each tape administration 
package to assure uniformity in 
administration. Instructions are 
played back from the tape recording to 
prevent reading difficuJties from 
interfering with an incJividuars 
ability to respond. Includes response 
time. 



parental education. The leve! of 

education of the mother and father cf 
an a' Jessed student as derived from 
the student's response tc two 
assessment items. A Year 15 reporting 
subgroup. 



panicipant. See respondent. 



percent-correct. The estimated 

proportion of a target population who 
would answer a particular exercise 
correctly. 



OERl, Office for Educational Research 
and Improvement. 



0MB. Office of Management and Budget. 

open-ended response item, A 

non-multiple-choice exercise that 
requires some type written or o. il 
response. 



oversampling. Deliberately sampling a 
portion of the population at a higher 
rate than the remainder of the 
population. 



persuasive writing, A writing 

objective of the Year 15 assessment. 
Writing that aiienipts to breing about 
some action or change. 

plausible values. Proficiency values 
drawn at random from a conditional 
distribution of a NAEP respondent 
given his or her response to cognitive 
exercises and a specified subset of 
background variables (conditioning 
variables). The selection of a 
plausible value is a form of 
imputation. 



population. An aggregate of elements, 
usually individual units with 
associated characteristics for 
observation or measurement. 
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post-stratification. Classification of 
selected sampling units by a set of 
strata definitions after the sample 
has been selected. 



Probability Sample. A sample in which 
every element of the populaiion has a 
kncsvn. non-zero probability of being 
selected. 



PPS, Probability Proportional to Size. 

precision. The difference between the 
expected value and the sample estimate 
of a population value, as measured by 
the sampling error. 



Primary Sampling Unit, A primary 
sampling unit. This is the basic 
gcogn phic sampling unit for National 
Assessment. A PSU is either a single 
county or a set of contiguous 
counties. Sec also multi-stage sample 
design. 



primary trait scoring, A method of 

scoring open-ended response exercises 
by evaluating the ability to write for 
precisely defined purposes. Criteria 
for evaluating responses arc 
associated with specific point scores 
in a scoring guide. 

Principal Questionnaire, A data 
collection form given to school 
principals. The principals respond to 
questions concerning enrollments, size 
of the community, occupational 
composition of the community, etc. 



proportional allocation. Allocation of 
a sample to strata in proportion to 
observational units in each stratum 



pseudo-replicate. The value of a 

statistic based on an altered sample. 
Used by the jackknife variance 
estimator. 



PSU. See primary sampling unit. 



public-use data tapes. Computer tapes 
containing respondent-level cognitive 
item, background and altitude and 
demographic data. Avt,:iablc for use 
by researchers wishing to do secondary 
analyses of NAEP da\a. 



PUDT. See public-use data tapes. 



QED. Quality Education Data, Inc. A 
suplier of additional student and 
school data for Year 15. 



random ariable. A variable which takes 
on any value of a specified se: with a 
particular probab»U^y. 



Probability Proportional to Estimated Size 
(PPES). Selection method where 
probabilities of selection for 
sampling units are assigned in 
proportion ;o the magnitude of the 
estimated size measure for each unit 



reading proficiency scale. Scale (0 »o 

500) based on IRT upon which levels of 
reading performance can be measured. 



receipt control. Procedures used Dy 
scoring staff to check in and screen 
field materials. Information from 
these procedure.*: Is relayed to 
assessment administrative staff so 
that any errors may be corrected. 
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recycled exercises. The set of 

exercises that is kept secure from one 
assessment to the next that will be 
used to measure changes (growth, 
stability or decline) in performance 
for the learning area. 



region. One of four geographical 
regions used in gathering and 
reporting data: Northeast, Southeast, 
Centra' and West (as defined by the 
Office of Business Economics, U. S. 
Department of Commerce). A Year 15 
reporting subgroup. 



released item. An item for which 
results and item text have been 
reported to the public. 



reliability check. The scoring of 

open-ended response items by a second 
scorer. In Year 15, twenty percent of 
these items underA^ent reliability 
checks. 



reporting subgroups. Groups within the 
national population for which National 
Assessment data are reported: sex, 
race/ethnicity, grade, age, level of 
parental education, region, and size 
and type of community. 



rescore. If an open-ended exercise was 
scored under different conditions than 
presently held or if passage of time 
may affect scoring, responses from an 
earlier assessment may be rescored at 
the same time as responses from a 
later assessment. Responses from an 
earlier assessment also may be held 
and not scored so that they can be 
scored with responses from a later 
assessment. 
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Research Triangle Institute. The NAEP 
survey subcontractor prior to the Year 
15 assessment; drew the sample of PSUs 
and schools for the Year 15 
assessment. 



resolution mode. Processing option 

under the data entry system; used for 
the correction of erroneous or 
discrepant data values. 



respondent, A person who is eligible 
for National Assessment, is in the 
sample, and who responds by completing 
one or more items in an assessment 
booklet. 



response error. The difference between 
the observed value and the true value 
for an observational unit. 



response experience. Response rates 

observed in previous surveys which are 
used for planning purposes. 



response options. Different 

alternatives to a multiple-choice 
question that can be selected by the 
respondent. 



response rate. Proportion of sampling 
units for which responses or 
measurements are obtained. 



review conference. A conference held to 
review the objectives of a learning 
area to assure their acceptance as 
measures of the objectives by 
scholars, educators and lay persons or 
to review exercises for racial, 
ethnic, social or regional bias. 

RP uale. See reading proficiency 
scale 
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RTl. See Research Triangle Institute. 



sample design parameter, A population 
parameter or a survey parameter, such 
as an expected response rate, used in 
designing a sample. 



sample design. Specifications for 

selecting a sample plus specifications 
for processing the sample data to make 
estimates. See sampling plan. 



swnple size. The number of units in the 
sample. (See also average sample 
size.) 



sample survey. As opposed to a census, 
a data collection process whereby only 
a sample of the population is observed 
or measured. 



sample. A portion of a population, or a 
subset from a set of units* selected 
by some probability mechanism for the 
purpose of investigating the 
properties of the population. NAEP 
does not assess an entire grade/age 
population but rather selects a 
representative sample from the 
grade/age group to answer assessment 
items. 



sampling error. The error that occurs 
because only a sample of the 
population is observed. Measured by 
standard error and variance. 



swnpling frame. The list of sampling 
units from which the samp'e is 
selected. 



sampling plan. Set of specifications 
and procedures used to select a 
sample. See sample design. 



School Characteristics and Policy 
Questionnaire. A five-page 
questionnaire completed for each 
school by the principal or other 
* ofHciai; used to gather information 
concerning school administration, 
staffing patterns, English curriculum 
and student services. 



school district. Administrative unit of 
the public school system, usually 
involving a school system under a 
single district organization. 



school response rate. The response rate 
for a sample of scnools. (See 
response rate.) 



sconng guide. A guide for hand scoring 
an open-ended response item that 
specifies descriptive or diagnostic 
categories by giving definitions and 
example responses. 



second-stage sampling unit. See 
multistage sample design. 



Secondary Sampling Unit (SSU). See 
multi-stage sample design. 



secondary traits. Characteristics of a 
response to an open-ended exercise 
indicating the presence or absence of 
elements that are of special 
significance to the exercise. 



secure items. Items not release for 
public use, in order to be 
readministered in subsequent 
assessments to determine whether 
performance levels have increased, 
decreased or remained the same. 



selection probability. The probability, 
or chance, that a particular sampling 
unit has of being selected in the 
sample. 



5E5. See socioeconomic status. 



session. See assessment session. 



seventeen-year-old. One of the National 
Assessment target populations. For 
Year 15, defined as persons born from 
October 1, 1966 to September 30, 1967. 



sex. One of the NAEP reporting 

subgroups. Assessment results are 
reported for males and females. 



simple random sample. Process for 
selecting n sampling units from a 
population of N sampling units so that 
each sampling unit has an equal chance 
of being in the sample ;»nd every 
combination of n sampling units has 
the same chance of being in the sample 
chosen. 



single-length block, A group of 

assessment items, 14 minutes long, 
containing an average of 12 minutes of 
reading and writing exercises and two 
minutes of background and attitude 
questions. 



Size and Type of Co, -rv (STOC), One 
of the NAEP reporting '^''foups. 
dividing the communities . nation 
into seven groups based on 
other characteristics. 



size measure. Value of a variable used 
to determine the allocation of the 
sample to strata or used to assign 
selection probabilities to sampling 
units within a stratum. 



size stratum, A stratum based upon the 
value of the size measures for units 
placed in the same stratum; e.g., a 
stratum for the largest units. 



SMSA, See standard metropolitan 
statistical area. 



Socioeconomic Status (SES), For 

sampling, the lower SES portion of the 
population (approximately 20 percent) 
is considered a subpopulation to be 
sampled. 



SSU size measure. Measure of size for a 
secondary sampling unit (SSU). 



standard error. A measure of sampling 
variability for a statistic. Because 
of NAEP*s complex sample design, 
standard errors are estimated by 
jackknifing first-stage sample 
estimates. 



Standard Metropolitan Statistical Area 
(SMSA), An area defined by the 
federal government for the purposes of 
presenting general-purpose statistics 
for metropolitan areas. Typically, an 
SMSA contains a city of at least 
50,000 population plus adjacent areas. 

stem. The portion of an item that 
states the problem or asks the 
question. 



stimulus. For reading items, a visual 
stimulus used as part of the stem. 
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STOC. Sec size and type of community. 



stratification. The division of a 

population into parts, called strata. 



stratified sample. A sample selected 
from a population which has been 
stratified with part of the sample 
coming from each stratum. The strata 
may be either subdivisions of the 
population for which separate 
estimates are desired or subdivisions 
defined for the purpose of reducing 
sampling error. 



student frame. List of 

grade/age-eligible students within a 
sample school. 



student ID number. A unique 

identification number assigned to each 
respondent to preserve his or her 
anonymity. NAEP does not record the 
names of any respondents. 



student response rate. The response 
rate for a sample of students. See 
response rate. 



study skill item. An item requiring a 
special learned skill beyond the 
facility of recognizing and 
understanding the printed word: for 
example, the interpretation of a bar 
graph, telephone bill or table of 
contents. 



subgroup. See reporting subgroup. 



subject areas. See learning areas. 



subpopulation. See reporting subgroup. 



subswnpling. Selection of a sample from 
a larger sample. Also used to 
describe multi-stage sampling. 



subsegmenting. Operation of subdividing 
the area of a segment into several 
subareas and selecting one of the 
subareas. 



survey design. All specifications and 
procedures involved in a survey. 



survey population. The population 

actually surveyed or represented by 
the sample. May differ from the 
target population. 



systematic sample (systematic random 
sample). A sample selected by a 
systematic method; for example, when 
units are selected from a list at 
equally spaced intervals. 



TAC. See Technical Advisory Committee. 



tapescript. A script prepared for the 
announcer to use in producing the 
paced tape, indicating exactly what is 
to be read or not read aloud to the 
students as well as the amount of 
response time allowed for each 
exercise. See paced tape. 



target population. Same as population. 



Teacher Questionnaire. A nine-page 
questionnaire completed by selected 
English and Language Arts teachers: 
used to gather information concerning 
year of teaching experience, frequency 
of writing assignments, teaching 
materials used, and the availability 
and use of computers. 
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Tevnnical Advisory Committee, Committee 
of experts in areas of educational 
policy and procedures, mathematics, 
and measurement theory; provides 
advice and recommendations concerning 
NAEP staff technical plans such as 
sampling, program implementation and 
analyses. 



ihela scale. A reseating of the 
Reading Proficiency 5ca!e that 
standardizes the combined age and 
grade samples. Item response theory 
calculations are carried out in the 
theta scale for mathematical 
convenicjnce, then transformed to the 
Reading Proficiency scale for 
reporting purposes. 

third-stage sampling unit. See 
multistage sample design, 

thirteen-year-olds. One of the National 
Assessment target populations. For 
Year 15, defined as persons born 
during calendar year 1970. 

T-unit, Used to assess the quality of 
syntax used in an essay; an 
independent clause and all of its 
modifying words, phrases and clauses. 



UBIB (UnBalanced Incomplete Block) 

spiralling. Refers lo a portion of 
the spiral design in which each 
booklet contains a common block of 
background questions, a single-length 
block of assessment exercises, and a 
double-length block of assessment 
exercises. 



unequal probability sampling, A sample 
selection procedure in which the 
sampling units have assigned selection 
probabilities which are not equal for 
all units. 
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user tapes. See public-use data tapes. 

variance. The square of the standard 
error: the average of the squared 
deviations of a random variable from 
the expected value of the variable. 



verification mode. Processing option 
under the data entry system; used for 
the substantiation of data values. 



WARM. See Weighted Average Response 
Method. 



weight. A multiplicative factor equal 
to the reciprocal of the probability 
of a respondent being selected for 
assessment with adjustment for 
nonresponse; an estimate of the number 
of persons in the population 
represented by a respondent in the 
sample. The sum of weights for all 
respondents at an age level is an 
estimate of the number of persons in 
the country at that age level. 

Weighted Average Response Method (WARM). 
A generalization of the Average 
Response Method allowing the 
estimation of weighted averages. 

Westat. Inc. The NAEP survey 
subcontractor for the Year 15 
assessment. 



Winsorizing. Replacement of data 

values which are more extreme than a 
given threshold by that threshold 
value. Bounds the influence of 
extreme data values on an estimator 
while maintaining information on the 
sign of the values. 



writing scale. Scale based on Average 
Response Method upon which levels of 
writing performance can be measured. 



Year 01, 02, 03.,. 15, A sequential 
number assigned to each period of 
assessment activities in the field. 
Year 15 prc-assessmcnt activities 
began in May 1983; assessment 
activities began in August 1983 and 
ended in May 1984. 



